Opened 4 years ago

Closed 4 years ago

Last modified 2 years ago

#2003 closed defect (invalid)

regex generated variable loses character encoding when used in proxypass rule

Reported by: excudo@… Owned by:
Priority: minor Milestone:
Component: documentation Version: 1.19.x
Keywords: Cc:
uname -a: Linux [[removed the host]] 2.6.32-696.1.1.el6.x86_64 #1 SMP Tue Apr 11 17:13:24 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.12.0
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-17) (GCC)
built with OpenSSL 1.0.1e-fips 11 Feb 2013
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'

Description

I had exactly the same problem as is described in this stackoverflow post:

https://stackoverflow.com/questions/8838905/how-to-avoid-nginx-to-replace-20-by-whitespace-when-using-as-a-proxy-proxy-pas/31440150#31440150

The proxypass was towards tomcat and in those logs I could see that everything after the (originally encoded) space went missing. Even the HTTP/1.1 part, which caused it to throw a 505.

I can confirm the workaround mentioned in the answers (to assign it to a new variable) works. But this should be fixed structurally, without the need for a workaround.

Change History (3)

comment:1 by Maxim Dounin, 4 years ago

Resolution: invalid
Status: newclosed

The stackoveflow post uses a part of the unencoded URI as used in location matching. Quoting docs:

The matching is performed against a normalized URI, after decoding the text encoded in the “%XX” form, resolving references to relative path components “.” and “..”, and possible compression of two or more adjacent slashes into a single slash.

That is, a configuration like:

location ~ /api/search(/.*) {
    proxy_pass http://127.0.0.1:3003$1;
}

is expected to produce incorrect requests to proxied server. One is expected to use properly encoded variables when using variables in proxy_pass, and this might not be trivial to do. In this particular case, a better solution might be to use a much simpler configuration without variables:

location ~ /api/search/ {
    proxy_pass http://127.0.0.1:3003/;
}

In more complex cases using rewrite ... break; followed by a proxy_pass without an URI component is usually the simplest solution.

Note well that the "workaround" with using set mentioned on the stackoverflow is actually a misuse of a bug (#348). Avoid using it.

comment:2 by JoshMcCullough@…, 2 years ago

I commented on this other ticket, which is for the same issue. IMO the NGINX team should really consider at least / only encoding spaces in the request-line which proxy_pass generates.

Alternatively, provide us with a built-in directive, e.g. encode_uri:

location ~ /api/search/(?<path>.*) {
    encode_uri $path $encoded_path 
    proxy_pass http://127.0.0.1:3003/$encoded_path;
}

in reply to:  2 comment:3 by Maxim Dounin, 2 years ago

Replying to JoshMcCullough@…:

I commented on this other ticket, which is for the same issue. IMO the NGINX team should really consider at least / only encoding spaces in the request-line which proxy_pass generates.

As already replied in #1930, only encoding spaces is certainly wrong thing to do.

Alternatively, provide us with a built-in directive, e.g. encode_uri:

location ~ /api/search/(?<path>.*) {
    encode_uri $path $encoded_path 
    proxy_pass http://127.0.0.1:3003/$encoded_path;
}

See ticket #52.

Note: See TracTickets for help on using tickets.