Optimising for High Latency Environments – Web Performance and Site Speed Consultant

16 September, 2024

Written by Harry Roberts on CSS Wizardry.

Last week, I posted a short update on
LinkedIn
about CrUX’s new RTT data. Go and give it a quick read—the context will help.

Chrome have recently begun adding Round-Trip-Time (RTT)
data
to the Chrome User Experience Report (CrUX). This gives fascinating insights
into the network topography of our visitors, and how much we might be impacted
by high latency regions.

What is RTT?

Round-trip-time (RTT) is basically a measure of latency—how long did it take to
get from one endpoint to another and back again? If you’ve ever pinged
www.google.com over in-flight wifi, you’ve measured RTT.

Latency is a key limiting factor on the web: given that most assets fetched by
webpages are relatively small (compared to, say, downloading a software update
or streaming a movie), we find that most experiences are latency-bound rather
than bandwidth-bound.

The round trip also measures intermediate steps on that journey such as
propagation delay, transmission delay, processing delay, etc. These
intermediates fall outside of the scope of this article, but if you’ve ever run
a traceroute, you’re on the right lines.

Where Does CrUX’s RTT Data Come From?

RTT is designed to replace Effective Connection
Type
(ECT) with higher resolution timing information. To this end, it’s important to
realise that RTT data isn’t a measure of visitors’ latencies to your site, but
a measure of their latencies period. RTT is not a characteristic of your site,
but a characteristic of your visitor. It’s no different to saying this person
was from Nigeria or this person was on mobile or this person was
on a high latency connection.

You can’t change that someone was from Nigeria, you can’t change that someone
was on a mobile, and you can’t change their network conditions. RTT isn’t
a you-thing, it’s a them-thing.

RTT data should be seen as an insight and not a metric. If you find that you
have a large number of users on high latency connections, you need to build your
applications sympathetically. That’s exactly what this article is about.

How Can I See RTT Information?

As the inclusion of RTT data is still in its infancy, viewing it isn’t yet as
straightforward as other CrUX insights. However, there are a handful of
ways available to us—some are, admittedly, more easy and free than others.

CrUX API

To see the 75^th percentile RTT data for a given origin, you could use
the CrUX API:

curl "https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=" 
      --header 'Content-Type: application/json' 
      --data '{"origin": "https://website.com", "formFactor": "DESKTOP", "metrics": ["round_trip_time"]}'

…replacing , https://website.com, and DESKTOP with the relevant
inputs. For my site, I can see that my mobile RTT stands at 144ms and my desktop
RTT is 89ms—a difference that I can’t imagine we’ll find surprising.

Treo

If you don’t yet have a Treo account, you’re seriously
missing out. Go and sign up. It’s a magical tool that
makes life as a performance engineer so much easier (and much more fun). Treo
has begun adding RTT data at the URL level, which is incredibly exciting:

Again, because RTT is a characteristic and not a metric, Treo does the smart
thing and includes it in the Devices dashboard and not in, say, the Loading
dashboard.

Tame the Bots

Dave Smart has built a great CrUX History
visualiser
over on his site Tame the Bots—you can go play around with it there
and see both origin- and URL-level CrUX data, including the new RTT.

One particularly nice touch is his plotting RTT against TTFB—first-byte time
includes one round trip,
remember.

Improving Experiences for High Latency Environments

Before we dive in, I want to reiterate that this article is about general
approaches to optimising high-latency experiences—it is not about improving
metrics within the CrUX dataset. What follows is overall best-practice advice
for designing with latency in mind.

This section details opportunistic upgrades we can make that will hopefully
improve latency-bound visitors’ experiences.

Reduce Transfer Size

Broadly simplified…

Web servers don’t send whole files at once—they chunk them into packets and send
those. These are then reassembled on the client. Each of these packets has its
own RTT lifecycle (although not necessarily synchronously). This means that
larger files that require more packets will incur more round trips—each round
trip is latency. The speed at which files download will be a function of
bandwidth and round trip time.

If you want resources to load faster on high-latency connections, making them
smaller is still a sensible idea, although file size typically correlates more
with available bandwidth as file sizes increase.

Use a CDN

One of the most effective ways to reduce round trip times is to reduce the
distance itself. I have a client in Prague who also hosts their site on-prem in
the same city. They don’t currently have
a CDN, yet they do
experience high traffic levels from all over the globe:

Looking at their popularity rank, they’re more popular in certain sub-Saharan
countries than they are in their own home country of Czechia! Getting this
client set up on a CDN (probably
Cloudflare) is one of my top
priorities for this project.

As well as offering a whole host (ahem…) of other performance and security
functionality, the primary benefit of using a CDN is simply geographic
proximity. The less distance data has to travel, the faster it will get there.

If you aren’t using a CDN, you should be. If you are, you probably get some or
all of the next sections for free anyway…

Use a Fast DNS Provider

One of the first things a new visitor will have to do to access your site is
resolve the IP address using the Domain Name System (DNS). As a website owner,
you have a degree of control over who you use as your authoritative provider.
Cloudflare manages my DNS, and they’re among the
fastest. If possible, make sure you’re using someone
who ranks highly.

Upgrade to HTTP/2

Over 75% of responses served on the web are sent over
HTTP/2, which is
great! If you are one of that remaining 25%, you should prioritise it. By moving
to a CDN, you’re likely to get HTTP/2 as standard, so that’s two birds with one
stone.

A key benefit of HTTP/2 over HTTP/1.1 is better connection utilisation, which
results in reduced overall connection negotiation.

HTTPs 1 and 2 both run over Transmission Control Protocol (TCP). When two HTTP endpoints want to communicate, they need to
open a connection by way of a three-way handshake. This is almost all pure
latency, and should be avoided where possible.

If we take my site’s current 144ms mobile round trip, opening a TCP connection
would look like this:

The TCP would more accurately be a combination of SYN and ACK, but
that’s beyond the scope of what I’m trying to illustrate in this
article.

One whole round trip (144ms) before I can dispatch a GET request for a page.

An inefficiency present in HTTP/1.0 was that a connection could only satisfy one
request–response lifecycle at a time, meaning fetching multiple files (as most
webpages require) was a very slow affair.

To mitigate this, HTTP/1.1 permitted the opening of multiple simultaneous
connections to a server at once. This number did vary, but is colloquially
agreed to be six. This meant that a client (e.g. a browser) could download six
files at a time by opening six connections. While this was overall faster, it
introduced six times more cumulative latency by opening six separate TCP
connections. One saving grace was that, once the connection was opened, it was
kept open and reused (more on this in the next section).

You can visualise loading my homepage over an HTTP/1.1 connection below. Each of
DNS, TCP, TLS can be considered pure latency, but I’m only
talking about TCP right now.

Note that we open five connections to csswizardry.com, six to
res.cloudinary.com, and 23 TCP connections in total: that’s a lot of
cumulative latency! However, notice that the connections are reused (again, more
on that in the next section).

HTTP/2’s solution was to only open one TCP connection, greatly reducing the
connection overhead, and allow many concurrent downloads by multiplexing streams
within that connection:

Now we only have two connections to csswizardry.com (one needed to be CORS
enabled),
one to res.cloudinary.com, and 13 in total, all reused. Much nicer!

HTTP/2 reduces the amount of overall latency incurred by not having to navigate
lots of new or additional three-way handshakes.

A Word On HTTP/1.0

HTTP/1.0 is such a legacy protocol that I only really want to bring it up here
as a piece of trivia. I truly hope no one reading is running over HTTP/1.0.

In HTTP/1.0, the problem was compounded by the fact that connections were
immediately closed after use. This meant that every single file would need its
own connection negotiating. Every single file incurred a whole bunch of use-once
latency:

Each response has its own connection that gets immediately terminated. It really
doesn’t get much slower than that.

Key Takeaway

Upgrade to HTTP/2, and ensure that any connections you do have to open are
reused and persistent.

Upgrade to TLS 1.3

Hopefully you noticed something in the previous section: the connection was
insecure. I briefly mentioned DNS earlier, and
we looked a lot at TCP, so now it’s time to look
at TLS.

In the terrifying case you are running HTTP and not HTTPS, get that fixed as
a matter of urgency.

If we upgrade to HTTP/2, we have to also be running HTTPS—it’s part of the
requirements. It’s safe to assume, therefore, that if you’re running HTTP/2,
you’re also running securely. That does mean more latency, though…

This is now three round trips (432ms) before I can dispatch a GET request!
The additional layer of security is added onto the end of the TCP connection, meaning further round trips. I’d rather have
a secure site than a fast one, but if I could really choose, I’d choose both.

Simply by upgrading to TLS 1.3, we get access to
built-in optimisations. TLS 1.3 cuts out an entire
round trip by having removed some legacy aspects of the protocol:

Now it’s two round trips (288ms) before I can dispatch a GET request.
Faster. But not exactly fast. Let’s keep going.

TLS 1.3+0⁠-⁠RTT

An additional, optional feature of TLS 1.3 is
0⁠-⁠RTT for resuming previous connections. By sharing a Pre-Shared Key (PSK) in
the first handshake, we can send a GET request at the same time:

Now our GET request is dispatched after one round trip (144ms)!

Because of security trade offs, 0⁠-⁠RTT is an optional mechanism in TLS 1.3.

Key Takeaway

Security is vital, but it doesn’t have to be slow. Switch over to TLS 1.3 to get access to reduced round-trips on new
connections, and potential zero round-trips on resumed connections!

Upgrade to HTTP/3 (QUIC)

By upgrading to HTTP/3, what we’re really getting access to is QUIC. HTTPs 1 and
2, as discussed, are built on top of TCP. HTTP/3 is built on top of QUIC, which
implements a TCP-like layer on top of the inherently much faster UDP protocol.
It’s all the safety and properness of TCP, but avoiding many of its latency
issues. All of these changes and improvements are abstracted away from the
day-to-day developer, and you do not need to alter your workflows at all, so
I won’t elaborate on the differences between HTTP/2 and 3, or between TCP, UDP,
and QUIC in this article.

I will say, though, that it breaks my heart that the pure elegance,
time, and effort that has gone into protocol design is largely lost on end-user
developers. We simply flick a switch somewhere and all of this stuff Just
Happens™. We really don’t deserve it, but I digress…

That said, one of the key improvements in HTTP/3 is that, because it’s built on
top of QUIC, which in turn has the benefit of access to the transport layer, it
is able to provide TLS as part of the protocol. Instead of happening after our
initial connection, it happens as part of it!

Our GET request is now dispatched after just one round trip (144ms)!

Here is a neat example of observing the parallelisation in DevTools: note that
Initial connection and (the incorrectly labelled) SSL are parallelised and
identical:

This means that HTTP/3’s worst-case model mimics TLS 1.3+0⁠-⁠RTT’s best case. If
you have access to HTTP/3, I would recommend switching it on.

QUIC 0⁠-⁠RTT

Not to be confused with, but because of, TLS 1.3+0⁠-⁠RTT, QUIC also has its own
0⁠-⁠RTT
model.
This is a result of QUIC folding TLS into the protocol itself. This cumulative
effect of new protocol-level features means that resumed HTTP/3 sessions can
make use of a 0⁠-⁠RTT model to send subsequent requests to the relevant origin:

Now, our request is dispatched after zero round trips (0ms). It doesn’t
GET (heh…) faster than that.

Connection Migration

As if to make all of this even more impressive, QUIC gives us access to
Connection Migration! The bad news?

No one currently implements it, but when they do…

Internet users, particularly on mobile, will experience
changes in network conditions throughout their browsing lifecycle: connecting to
a new cell tower as they walk through a city; joining their own wifi connection
after arriving home; leaving a wifi connection when they leave a hotel.

Each of these changes would force TCP to negotiate brand new connections. TCP
uses a four-tuple method to keep connections in sync, whereby the client’s IP
address and port, plus the server’s IP address and port, are used to identify
a connection. Any change in any of these four parameters would require a new TCP
connection to be opened.

QUIC specifically designed its way around this by utilising a Connection ID to
identify open connections, leaving it immune to changes in any of the four
tuples. This, again, is thanks to QUIC being a ‘clean slate’ protocol.

This means that, rather than having to completely tear down and rebuild any
current connections due to a network change, in our best-case scenario, HTTP/3
can seamlessly resume on an existing connection. That looks like this:

In an H/3 world, the worst case scenario is a one-round-trip connection. That’s
a pretty great worst case:

If we were still running a TCP-based protocol such as HTTP/1 or 2, our
best-case scenario would resemble a TCP 1.3+0⁠-⁠RTT setup:

Our worst case would likely be an HTTP/1 or 2 over TLS 1.2 scenario:

Tear everything down; do everything again.

Key Takeaway

HTTP/3’s underlying protocol, QUIC, is able to fold TLS into its design by
default, eliminating the need to perform connection and TLS back-to-back. It can
also provide genuine seamless connection migration as devices traverse the
internet.

Avoid Incurring Latency

Alright! They were all fairly opportunistic upgrades, but what happens if a) you
can’t upgrade your protocols or b) you’ve already upgraded everything you can?
The best option, always, is to avoid. Prevention, as they say, is cheaper than
the cure. How can we side-step latency entirely?

Avoid Unnecessary New Connections

Avoiding too many HTTP requests was sound advice in an HTTP/1.1 world, where
requests and connections were inherently limited. In the HTTP/2 world, we’ve
been told we can take a slightly more carefree approach. However, where
possible, avoiding unnecessary connections is still fairly wise.

Where possible, avoid going to third-party origins especially for anything on
the Critical Path. I’ve said it before, and I’ll say it again and again until
everyone listens: Self-Host Your Static Assets.

This client of mine has a huge gulf between TTFB and First Contentful Paint, and
a huge contributor to that is time lost to latency—negotiating new connections,
many of which are unnecessary and on the Critical Path (denoted by ):

Looking at the CrUX data, their visitors’ RTT times are in line with the slowest
25% of RTT times globally—this is a client who needs to optimise for latency. By
self-hosting the majority of these resources, we can immediately regain a lot of
ground.

Key Takeaway

Although connections aren’t as scary as they used to be, setting new connections
up is pure latency—avoid doing so, particularly on the Critical Path.

Avoid Redirects

Where at all possible, avoid redirects. Redirects are also pure latency. I’ve
seen scenarios before where developers author all of their hrefs to point at
a non-trailing slash, e.g.:

…but their site’s URL policy contains a trailing slash, e.g.:

https://wwww.website.com/products/

This means that very link click a user makes will incur a full round trip of
latency in order to be served a 3xx-class redirect, which will then incur more
round trips to access the resource listed in the Location header:

I’d recommend looking into how many 3xx-class responses you serve—I’ve had
a number of clients this year alone who were, unbeknown to them, losing an
inordinate amount of time to redirects!

Interestingly, 304 responses are still a form of redirect: the server is
redirecting your visitor back to their HTTP cache. Ensure you aren’t wastefully
revalidating still-fresh
resources:

The act of redirecting from http to https is very much mandatory and should
always be carried out regardless of any time penalty, but this can be sped up by
using HSTS, which we’ll cover shortly.

Key Takeaway

While sometimes unavoidable, redirects are also pure latency. Ensure you’re not
causing unnecessary work, and tell your marketing department to stop using URL
shorteners.

Avoid Preflight Requests

Non-simple HTTP requests are automatically prepended by pure-latency preflight
requests. Preflight requests are issued when the actual request meets certain
CORS conditions, such as emitting a non-standard request header, or attempting
to make a DELETE request, for example.

This is a common source of latency in single page apps that hit API endpoints.
Take this client for example: the requests to their API endpoint carry
a non-standard Accept-Version header. This automatically kicks off a preflight
so that the server is made aware of the incoming request and has an opportunity
to reject it.

The above preflight OPTIONS requests are made with the following request
headers (formatted for neatness):

Origin:                         https://website.com
Access-Control-Request-Method:  GET
Access-Control-Request-Headers: Accept-Version

The server responds to the preflight request with a 204 containing the
corresponding response headers (formatted for neatness):

Access-Control-Allow-Origin:  https://website.com
Access-Control-Allow-Methods: HEAD,
                              GET,
                              POST
Access-Control-Allow-Headers: Accept-Charset,
                              Accept-Encoding,
                              Accept-Language,
                              Accept-Version,
                              Authorization,
                              Cache-Control,
                              Content-Type,
                              Server-Id

This tells the browser that https://website.com is allowed to make requests of
the listed method types, and with the listed headers.

Once this has taken place—all pure latency—the browser can finally make the
actual request which carries an Accept-Version: 1.0 that the earlier preflight
had asked about.

Where possible, avoid making non-simple requests, as doing so will trigger
a preflight that is pure latency. The conditions in which a request would
trigger a preflight request are listed on
MDN.

If you can’t avoid making preflight requests, read on.

Key Takeaway

If you’re building an SPA (which you probably are (and you probably shouldn’t
be)), check what’s
happening with your client-side API calls.

Pay Latency Costs Up-Front and Out of Band

Even with the best will in the world, we will have to incur some latency.
Techniques like 0⁠-⁠RTT only work for resumption, and hitting no other
origins whatsoever is virtually impossible. So can we pay the latency cost
up-front?

preconnect

We can use preconnect (sparingly) to preemptively open up connections to
important origins we’ll need to visit soon. I’ve written about configuring
preconnect before, so I’d
recommend giving that a read.

preconnect is a hint that the browser is going to need to open a new
connection to the supplied origin, and divorces the setup cost from the
initiating request:

 rel=preconnect href=https://fonts.gstatic.com crossorigin>

That gives us this nice shift left in the waterfall:

Generally speaking, you’d only want to preconnect any origins that are
important to the page (Google Fonts, yes; Google Analytics, no) and things that
aren’t referenced early in the already. Bonus points for deploying
preconnect as an HTTP
header
or Early Hint!

Speculation Rules API

One step further than preconnecting the origin is to actually preemptively
fetch the resource itself using either of prefetch or prerender in the new
Speculation Rules API. This mechanism allows us to pay any latency penalties
ahead of time and behind the scenes, so by the time a user clicks through to
their next page, it’s hopefully already fetched and waiting.

I wrote about this
recently,
so again, I’ll point you to that, but remember to tread carefully. With things
like preconnect, prefetch, preload, and prerender, less is always more.

Cache Everything

If you’re going to do something, try only do it once.

In the event we can’t make the relevant upgrades, and we simply can’t avoid
incurring latency, then we’d better try really hard to cache the results of any
latency-bound interactions…

HTTP/Browser Cache

The fastest request is the one that’s never made. Ensure you have a solid
caching (and revalidation) strategy in place. I’ve
written and
spoken at length about
HTTP cache so you can get everything you’ll need (and more…) from there.

CDN-Level

CDNs only help solve latency if requests terminate there: anything that gets
passed back to origin will remain on the slow path.

To fully maximise the benefits, ensure your CDN is configured to fully leverage
edge-level caching. If you need to set CDN (or shared) cache values separately
to your browser cache, use the s-maxage Cache-Control
directive.

Strict Transport Security

The first time someone hits your site over http, they’re likely (hopefully)
going to get redirected to https. If you opt into using HTTP Strict Transport
Security
(HSTS), then you can get the browser to cache this redirection on their end,
meaning you don’t incur a latency-bound 3xx-class to nudge the visitor over to
your secure URLs in future.

HSTS is deployed by way of a Strict-Transport-Security response header, e.g.:

Strict-Transport-Security: max-age=31536000

Not only is this faster, it’s more secure.

To get even faster and even more secure, you can get your site added to the
HSTS Preload list. This hard-codes your origin(s)
into the browser so that there is never a first time http to https 3xx
redirect: you’ll never incur that latency (or exposure), not even once.

Cache Your Preflights

As before, if you can’t remove your preflight requests, you can at least cache
them. This works differently to your usual Cache-Control header, and is
implemented with the dedicated Access-Control-Max-Age response header. Give
its value serious consideration—this is an important security-facing feature. To
stop developers being too permissive, Firefox limits us to a maximum 24 hours
and Chrome to just two—even if you passed in 31,536,000 seconds (one year), the
best you’d get is 86,400 (one day):

Access-Control-Max-Age: 86400

These headers, much like any response header, are per-URL, so you can’t set an
origin-wide policy (which is a feature, not a bug).

Key Takeaway

Any latency that can’t be avoided, take the hit once and deal with it.
Subsequent occurrences should be mooted by virtue of being cached.

So, What Are My Options?

You have lots of options, but do remember that I just spent almost 5,000 words
explaining how to solve what may be your least severe liability. Only if you
know, and it’s very apparent, that latency is your biggest killer, should you
embark on most of the items in this article.

My first recommendation would be to contain as many of your current problems as
possible by aggressively caching anything expensive.

Next, work to avoid anything that you could subtly rework or refactor—it’s
better to not do it at all, if we control it.

Things that can’t be avoided, attempt to solve out of band: preconnecting
origins, or prerendering subsequent navigations are real quick wins.

Beyond that, look to opportunist upgrades to get yourself ahead of the curve.
Protocol-level improvements can swallow a lot of preexisting issues for us.

However, a lot of the things I have discussed are either:

1. trivial to implement just by using a decent CDN, and;
2. best practice anyway.

Appendix

If you’re interested in comparing the different protocol-level differences
side-by-side:

Many thanks to Barry Pollard and Robin
Marx for feedback and input on this article

Specs for the protocols discussed can be found at: