Opened 9 months ago

Closed 11 days ago

#2349 closed enhancement (fixed)

Support range requests in requests served via gzip_static

Reported by: CAFxX@… Owned by:
Priority: minor Milestone:
Component: nginx-module Version:
Keywords: Cc:
uname -a: Linux 02dfaaf22c02 5.10.104-linuxkit #1 SMP Wed Mar 9 19:05:23 UTC 2022 x86_64 Linux
nginx -V: nginx version: nginx/1.21.6
built by gcc 10.3.1 20211027 (Alpine 10.3.1_git20211027)
built with OpenSSL 1.1.1l 24 Aug 2021 (running with OpenSSL 1.1.1n 15 Mar 2022)
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --with-perl_modules_path=/usr/lib/perl5/vendor_perl --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-Os -fomit-frame-pointer -g' --with-ld-opt=-Wl,--as-needed,-O1,--sort-common

Description (last modified by CAFxX@…)

As discussed in https://forum.nginx.org/read.php?2,209738,210053#msg-210053 nginx currently does not support range requests in responses that include Content-Encoding: gzip, the valid argument being that the industry-wide misuse of Content-Encoding: gzip in place of the more appropriate Transfer-Encoding: gzip for responses that are gzip-compressed on the fly makes it exceedingly hard to be able to satisfy such range requests.

It is worth noting though that this argument does not apply to responses served by gzip_static, as in this case the content is actually already gzip-encoded, and the length and ETag of the encoded content are known, and therefore satisfying the HTTP semantics of range requests + content-encoding is not difficult.

I would therefore suggest that, for responses that gzip_static can satisfy, the gzip_static module should add Accept-Ranges: bytes to the response (in addition to the Etag/Last-Modified/Content-Encoding headers that are already added), and add support for handling the Range: <ranges> and If-Range: <etag>/<lastmod> request headers in accordance with the HTTP spec (i.e. serve byte ranges interpreted as referring to the gzip-encoded resource; if an If-Range header is present in the request and it matches the current gzip-encoded resource, send 206 and the requested range, otherwise send 200 and the full gzip-encoded resource).

If we are worried this may break some non-compliant clients, it may be worthwhile to add an option to enable/disable range requests in gzip_static.

Change History (4)

comment:1 by CAFxX@…, 9 months ago

Description: modified (diff)

comment:2 by Maxim Dounin, 5 weeks ago

With Last-Modified being used as a validator, ranges with gzip_static might be still problematic: if a resource is re-gzipped with different settings, the Last-Modified time will be the same (considering gzip --keep behaviour, as well as gzip_static documentation, which recommends to keep the same modification time of the original and the compressed files), though binary representation will be different.

There are at least two mitigation factors though:

  1. Since version 1.3.3, released in 2012, nginx generates entity tags by default, and entity tags include not only modification time, but also the file size. This should be enough to request ranges safely, since the file size is likely to be different.
  1. Even if entity tags are not used, the Content-Range header will include the response size, making it possible for clients to detect that the binary representation was changed.

Overall, I tend to think it is safe enough to enable ranges support for gzip_static.

Patch can be found here:

https://mailman.nginx.org/pipermail/nginx-devel/2023-January/5X37HGXYDHRF55C4YXYMERGFELWJMPHJ.html

Review and testing appreciated.

comment:3 by Maxim Dounin <mdounin@…>, 11 days ago

In 8120:c7e103acb409/nginx:

Gzip static: ranges support (ticket #2349).

In contrast to on-the-fly gzipping with gzip filter, static gzipped
representation as returned by gzip_static is persistent, and therefore
the same binary representation is available for future requests, making
it possible to use range requests.

Further, if a gzipped representation is re-generated with different
compression settings, it is expected to result in different ETag and
different size reported in the Content-Range header, making it possible
to safely use range requests anyway.

As such, ranges are now allowed for files returned by gzip_static.

comment:4 by Maxim Dounin, 11 days ago

Resolution: fixed
Status: newclosed

Fix committed, thanks for prodding this.

Note: See TracTickets for help on using tickets.