Static site implosion with Brotli and Gzip

We’ve been trying out Brotli compression in our server setup, and are getting amazingly fast page loads as a result. Brotli is much better at compression than Gzip. However, thoughtlessly cranking up the compression level will hurt response time. We figured that a well-balanced mixture of static and on-the-fly compression might just do the trick.

As front-end developers, we know that a great user experience begins with fast page loads. Therefore, we’re always looking for ways to get code from the server to the client more quickly. One way to achieve fast transfers from server to browser, is to keep file sizes small. That’s where compression plays a key role.

Brotli is a new compression algorithm engineered by Google. Google introduced Brotli in 2015 and added it to its Chrome browser March 2016. Brotli is also supported by Firefox, Opera and in the Android browser.

Brotli does a better job compressing resources than its predecessor, Gzip. Cloudflare wrote an excellent article (with benchmarks) about Brotli compression compared to Gzip.

Capable browsers advertise their ability to accept Brotli-compressed resources in the Accept-Encoding request header.

If both browser and server support Brotli, the server uses Brotli compression and sets the Content-Encoding header of the response to br accordingly.

Finding this header in the response means that the resource has been compressed with Brotli!

Brotli is not yet mainstream in web server software. Getting your server ready for Brotli compression involves (re)compiling your server to include a Brotli module. There is a Brotli module for Apache as well as a Brotli module for Nginx.

At De Voorhoede, we are in the process of moving towards serving our website over Nginx in a container based architecture. We picked a Docker server image with Brotli compiled into Nginx, saving us the trouble of having to compile Nginx ourselves.

Configuring Nginx to use Brotli compression is not difficult once you have the module installed.

The brotli_comp_level determines the degree of compression, which ranges from lowest 1 to highest 11 and defaults to 6. You use the Brotli directive to enable the module and provide the mime types that you want to enable compression for. As excited as we were to put this new performance enhancement to the test, we started out with the following setup.

This turned out not to be such a great idea. Sure, the reduction in file size will be impressive on the receiving side. As it appears, however, it takes the web server quite a bit of time to get its response together with a compression level of this magnitude. As a result, we saw our time to first byte skyrocket for all resources. It took more than 1.8 seconds to load our home page’s first view. Not quite acceptable.

TTFB: 206.84 ms. 226.76 ms Total

As Tim Kadlec explains in his article about Brotli’s potential, The sweet spot of Brotli’s compression vs processor load, compared to Gzip, is at level 4. At that level, the level of compression is already higher than Gzip’s default, while also finishing compression more quickly. At higher rates, file sizes will get smaller, but the algorithm will also consume more processing power, causing it to take longer to get the job done.

So, what can we do to get maximum compression without hounding our web server CPU too much? Well, we do have a static site, so how about compressing resources before we deploy? Google’s Brotli module includes the following directive:

Enabling this will cause Brotli to look for files that have already been compressed and serve those files directly without performing any compression itself. What we also did, was to remove the brotli_typesdirective that instructs the server to compress all file types. This makes Brotli fall back to its default of only compressing the default mime type text/html. brotli_static then makes sure that Nginx returns the statically compressed version for any resource that it can match with a brotlified counterpart.

Getting the files ready to be served as statically compressed by brotli_static, is simple: In our build process, we added a build step that Brotli-compresses all static files using gulp-brotli. We don’t pre-compress HTML files, because on our website, pages are essentially dynamic. We serve an optimized version for first visits, which differs from the HTML that is served for repeat views. Read Declan’s article if you’re curious how that works.

Gulp-brotli can be instructed to skip compression of certain files that actually end up being larger than they were in uncompressed form. Woff2 files are already compressed with Brotli and several image types are already pretty well optimized during earlier build steps. If there’s nothing to gain from applying more compression, the plugin does nothing. Nevertheless, in most cases we do manage to shave off a few bytes here and there.

Of course, we didn’t forget about Gzip. Brotli is not yet for everyone, and the aforementioned optimizations also apply to Gzip, because a gzip_static also exists.

The relevant bit of Nginx configuration now looks like this:

These are the results for acceptance.voorhoede.nl:

This is the request/response timing from the Chrome DevTools network tab:

TTFB: 18.55 ms. 31.54 ms Total

TTFB is reduced from 207ms to 19ms. Much better!

We now have all static assets pre-compressed with Brotli and Gzip at the highest level, so that our server performance does not suffer. Our (dynamic) HTML is compressed on the fly with Brotli at level 4; a great trade-off between file size and server performance.