Opened 5 months ago
Last modified 5 months ago
#2567 new enhancement
sub_filter and gziped payload should trigger warning
| Reported by: | Owned by: | ||
|---|---|---|---|
| Priority: | minor | Milestone: | |
| Component: | nginx-core | Version: | 1.18.x |
| Keywords: | Cc: | ||
| uname -a: | Linux virtucon 6.2.0-1010-nvidia #10-Ubuntu SMP PREEMPT_DYNAMIC Tue Aug 29 17:57:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | ||
| nginx -V: |
nginx version: nginx/1.18.0 (Ubuntu)
built with OpenSSL 3.0.2 15 Mar 2022 TLS SNI support enabled configure arguments: --with-cc-opt='-g -O2 -ffile-prefix-map=/build/nginx-zctdR4/nginx-1.18.0=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now -fPIC' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-compat --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --add-dynamic-module=/build/nginx-zctdR4/nginx-1.18.0/debian/modules/http-geoip2 --with-http_addition_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_sub_module |
||
Description
Hi.
Reporting an issue that bit me quite some time ago. I don't have the setting to reproduce it because I solved it on my own and moved on. But here is the lesson learned:
I was busy reverse-proxying a website with a path prefix. (The kind of thing you do because you have only one machine, no control over DNS, and want to reuse the same port.) I encountered a Heisenbug because of default headers being passed around by the various tools I was using. But, bottom line:
The reverse proxying didn't work because I had a sub_filter declaration to correct the payloads to account for the prefix in the reverse-proxying. However, the backend gave gziped data. sub_filtering on compressed data is... doomed to failure.
So, for people encountering the same situation, and dealing with the same kind of Heisenbugs: please provide a warning in the logs for people attempting sub_filter on compressed data.
That would have helped me diagnose the problem MUCH MUCH MUCH faster, without having to dissect implicit behaviours related to compression.
In theory, the sub_filter is expected to work correctly with arbitrary data, including binary and/or compressed content. As such, I don't think that a runtime warning would be a good solution - or at least there should be a way to quench it.
On the other hand, docs probably should at least mention that compression on the backend might interfere with correct substitutions, and recommend to disable it as needed, either directly on the backend, or by using
proxy_set_header Accept-Encoding '';.Also, it is a long-planned feature in the gunzip module to make it possible to automatically gunzip content based on other modules being used. This looks directly related.