id,summary,reporter,owner,description,type,status,priority,milestone,component,version,resolution,keywords,cc,uname,nginx_version 1756,"Invalid or Binary in URI causes core 400 and 500 errors, plus binary in logfile",Co6aka@…,,"This is a complicated mess, so it'll take a while to explain... Background: I've been getting binary data in my logfiles because they include 'BODY=""$request_body""' and I've finally figured out why; hackers have been sending non-ascii characters in the request URI, but the same or similar results can be had by sending '%00' (percent-zero-zero) or '%%' in the request URI. When this happens NGINX is returning hardcoded ""internal"" error pages instead of my custom error pages, and after investing way too much time experimenting it seems there's no way to intercept this condition. Here's an example from the virtual server error log: 2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] 2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] 2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] 2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] 2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] 2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] 2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] 2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] 2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] 2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] 2019/03/29 20:16:13 [error] 28634#28634: *299 rewrite or internal redirection cycle while redirect to named location ""@errorpages"" while reading client request headers, client: 192.168.1.1, server: [my_servername_here] And here's the corresponding access log entry; note the ""-"" character at the end of the [servername] snip-out (for example, ""www.server.com-"") and that URI is """" instead of the typical ""-"" and that UA is ""-"" instead of the UA of my browser: [29/Mar/2019:19:55:58 -0400] 192.168.1.1 :57732 500 GET HTTP/2.0 :443 (170 bytes) REQ=""[my_servername_here]-"" URI="""" REF=""-"" USR=""-"" UA=""-"" The above example was obtained like this, with http or https... http://www.servername.com/%% http://www.servername.com/%00 I can trigger a status 500 that correctly returns my custom error page like this... location = /errpage.500 { return 500; } ...so I know MY error pages are working in the virtual server. You can also test it HERE like this... http://www.nginx.org/%00 https://www.nginx.org/%00 http://www.nginx.org/%% https://www.nginx.org/%% ...which returns the hardcoded internal error 400 page that includes ""400 Bad Request / nginx/1.15.7"" which is an information leak. There are no corresponding entries in the core error log, only in the virtual server log. Issues in no particular order are... 1: Binary data shouldn't be getting written into a text logfile; it should at least be in hex. 2: Invalid/binary in a URI shouldn't trip-up the core like this, and the URI is in fact NOT EMPTY even though this is reported in the logfiles. 3: It seems that the error condition occurs in the core, and is handled by the core, even though the virtual server's logfiles are being written to. 4: When custom error pages are configured in the virtual server the internal pages should not ""leak out"" to clients. 5: Perhaps there is or might be found some way to hack/crash NGINX this way??? Here is an example from a hack-attack, where I've hex-encoded the binary data from the log, which I had to read with a hex editor; the first character is ""0003"" in case it doesn't display here... 2019/03/28 04:05:57 [error] 22973#22973: *1100 empty URI in redirect to named location ""@errorpages"" while reading client request line, client: 141.98.81.34, server: [my_servername_here], request: ""[0003]\00\00/*\E0\00\00\00\00\00Cookie: mstshash=Administr"" There are many other hack-attacks with long strings of binary data; this is the shortest one.",defect,closed,minor,,other,1.15.x,invalid,,,Linux tux 5.0.5-050005-generic #201903271212 SMP Wed Mar 27 16:14:07 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux,"nginx version: nginx/1.15.8 (Ubuntu) built with OpenSSL 1.1.0g 2 Nov 2017 (running with OpenSSL 1.1.1b 26 Feb 2019) TLS SNI support enabled configure arguments: --with-cc-opt='-g -O2 -fdebug-prefix-map=/build/nginx-FWzBFO/nginx-1.15.8=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -fPIC' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_xslt_module=dynamic --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module"