Opened 5 years ago

Last modified 5 years ago

#1808 new defect

Inconsistent encoding in rewrites

Reported by: Timo Hoffmann Owned by:
Priority: minor Milestone:
Component: other Version: 1.16.x
Keywords: Cc:
uname -a: Linux localhost.localdomain 5.0.9-301.fc30.x86_64 #1 SMP Tue Apr 23 23:57:35 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.16.0
built by gcc 9.1.1 20190503 (Red Hat 9.1.1-1) (GCC)
built with OpenSSL 1.1.1b FIPS 26 Feb 2019
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-stream_ssl_preread_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-http_auth_request_module --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --with-ld-opt='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'

Description

Some characters get decoded in rewrites and can cause some trouble in other places like in proxy_pass.

Demo setup

location / {
   rewrite /foo/(.*) /bar/$1 redirect;
}

Exmple data

The URL ​http://localhost/foo/%5B%5D%7B%7D%C2%A7%24%25%26%3F will return the following location header from the webserver:
Location: ​http://localhost/bar/[]{}%C2%A7$%25&?

As you can see some chars (like []) got decoded but not all of them.
It seems that only the reserved characters of RFC 3986 will be decoded.

Change History (3)

comment:1 by zingaburga@…, 5 years ago

Bug still exists in 1.18.0.

It seems that it's only the first instance of %3f that gets decoded.

There's some suggestions around the internet to use the following workaround:

   rewrite /foo/([^?]*)\?(.*) http://localhost/bar/$1%3f$2 redirect;
   rewrite /foo/(.*) http://localhost/bar/$1 redirect;

...however that doesn't work for me. I noticed that replacing %3f with %%3f does make a difference for some reason.

Still, it doesn't really fix the issue - it seems that nginx just goes ahead and unescapes the next instance, e.g. ​http://localhost/foo/%3f%3f%3f redirects to ​http://localhost/bar/%3f?%3f instead.

If you can accept only handling a limited number of %3f characters, you can kinda work around the situation with the following awkward sequence:

   rewrite /foo/([^?]*)\?([^?]*)\?([^?]*)\?(.*) http://localhost/bar/$1%%3f$2%%3f$3%%3f$4 redirect;
   rewrite /foo/([^?]*)\?([^?]*)\?(.*) http://localhost/bar/$1%%3f$2%%3f$3 redirect;
   rewrite /foo/([^?]*)\?(.*) http://localhost/bar/$1%%3f$2 redirect;
   rewrite /foo/(.*) http://localhost/bar/$1 redirect;

(repeat the first line as many times as needed)

comment:2 by wknapik@…, 5 years ago

You can do this for redirects:

location / {
    rewrite ^ $request_uri;
    rewrite /foo/(.*) /bar/$1;
    return 301 $uri;
}

and this for proxying

location / {
    rewrite ^ $request_uri;
    rewrite /foo/(.*) /bar/$1 break;
    proxy_pass http://127.0.0.1:8080$uri;
}

I still think the decoding that nginx does makes no sense, but maybe I'm missing something.

In any case, if you need a solution that doesn't involve modifying nginx, the above should work.

comment:3 by wknapik@…, 5 years ago

See ​https://github.com/kubernetes/ingress-nginx/issues/5576 for a discussion of ~this problem in the context of the NGINX Ingress Controller for Kubernetes.

Note: See TracTickets for help on using tickets.