Invalid or Binary in URI causes core 400 and 500 errors, plus binary in logfile

[Co6aka@…]

This is a complicated mess, so it'll take a while to explain...

Background:

I've been getting binary data in my logfiles because they include 'BODY="$request_body"' and I've finally figured out why; hackers have been sending non-ascii characters in the request URI, but the same or similar results can be had by sending '%00' (percent-zero-zero) or '%%' in the request URI.

When this happens NGINX is returning hardcoded "internal" error pages instead of my custom error pages, and after investing way too much time experimenting it seems there's no way to intercept this condition. Here's an example from the virtual server error log:

2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]
2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]
2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]
2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]
2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]
2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]
2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]
2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]
2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]
2019/03/29 20:16:13 [error] 28634#28634: *299 empty URI in redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]
2019/03/29 20:16:13 [error] 28634#28634: *299 rewrite or internal redirection cycle while redirect to named location "@errorpages" while reading client request headers, client: 192.168.1.1, server: [my_servername_here]

And here's the corresponding access log entry; note the "-" character at the end of the [servername] snip-out (for example, "www.server.com-") and that URI is "" instead of the typical "-" and that UA is "-" instead of the UA of my browser:

[29/Mar/2019:19:55:58 -0400] 192.168.1.1 :57732 500 GET HTTP/2.0 :443 (170 bytes) REQ="[my_servername_here]-" URI="" REF="-" USR="-" UA="-"

The above example was obtained like this, with http or https...

​http://www.servername.com/%%
​http://www.servername.com/%00

I can trigger a status 500 that correctly returns my custom error page like this...

location = /errpage.500 { return 500; }

...so I know MY error pages are working in the virtual server.

You can also test it HERE like this...

​http://www.nginx.org/%00
​https://www.nginx.org/%00
​http://www.nginx.org/%%
​https://www.nginx.org/%%

...which returns the hardcoded internal error 400 page that includes "400 Bad Request / nginx/1.15.7" which is an information leak.

There are no corresponding entries in the core error log, only in the virtual server log.

Issues in no particular order are...

1: Binary data shouldn't be getting written into a text logfile; it should at least be in hex.

2: Invalid/binary in a URI shouldn't trip-up the core like this, and the URI is in fact NOT EMPTY even though this is reported in the logfiles.

3: It seems that the error condition occurs in the core, and is handled by the core, even though the virtual server's logfiles are being written to.

4: When custom error pages are configured in the virtual server the internal pages should not "leak out" to clients.

5: Perhaps there is or might be found some way to hack/crash NGINX this way??? Here is an example from a hack-attack, where I've hex-encoded the binary data from the log, which I had to read with a hex editor; the first character is "0003" in case it doesn't display here...

2019/03/28 04:05:57 [error] 22973#22973: *1100 empty URI in redirect to named location "@errorpages" while reading client request line, client: 141.98.81.34, server: [my_servername_here], request: "[0003]\00\00/*\E0\00\00\00\00\00Cookie: mstshash=Administr"

There are many other hack-attacks with long strings of binary data; this is the shortest one.

[Maxim Dounin]

1: Binary data shouldn't be getting written into a text logfile; it should at least be in hex.

Duplicate of #191.

As for your other questions, the problem is that you are trying to redirect errors which appear during early request processing stages - when the request URI is not yet parsed - to a named location. Named locations are designed to preserve existing request URI, and redirecting such an incomplete request to a named location is likely to cause serious problems in other parts of the code. As such, an attempt to do such a redirect results in the "empty URI in redirect to named location" error. This in turn prevents your custom error page from being used, so an internal one is returned.

If you want to handle errors which appear during early request processing stages, such as 400 Bad Request, consider using error pages with explicitly set URI. Or you may instead avoid trying to redirect such errors, as this is generally much safer approach.

Note well that the fact that server is nginx is not an information leak. It is anyway reported in a number of ways, including the Server response header. If you for ​some reason want to hide nginx version, you can do so via the ​server_tokens directive.

[Co6aka@…]

On the following page...
​https://www.nginx.com/resources/wiki/community/faq/
...it is written:
"What does this @ thing mean? @location is a named location. Named locations preserve $uri as it was before entering said location. They were introduced in 0.6.6 and can be reached only via ERROR_PAGE, post_action (since 0.6.26) and try_files (since 0.7.27, backported to 0.6.36)."

== AND ==

In the book "NGINX Essentials" by Valery Kholodkov, on page 59 it is written...
"Handling Errors ... A more sophisticated action can be taken ... using an error_page directive that points to a named location" followed by an example thereof.

== AND ==

Above it is written by mdounin: "Named locations are designed to preserve existing request URI" which is precisely what one would want when handling some error condition; an unmodified un-rewritten URI.

== THEREFORE ==

Since access to named locations is specifically tied to the error_page directive, NGINX being unable to gracefully handle "corrupt" URIs in this circumstance is almost as ridiculous as all the "Russians did it!!!" hysteria. SORRY, there is NO EXCUSE for not being able to safely and properly handle what is an error condition, namely a garbage-filled URI.

And to be even more precise, NGINX is reporting "empty URI in redirect to named location" when in fact the URI is NOT EMPTY; it's filled with garbage, which is an error condition.

BOTTOM LINE: Named locations are intended targets of error_page directives, thus code that handles named locations should properly handle error conditions at error-handling locations.

This is indeed an issue requiring a fix.

[Maxim Dounin]

Thank you for your opinion. As previously suggested, if you want to handle 400 errors, use error pages with explicitly set URI. Named error pages are not going to work if URI of a request cannot be parsed.

[Co6aka@…]

For a specific error page solution, something like this is required:

error_page 400 ​https://www.yoursite.com/errorpages/400.html;

OR, for a "universal" custom error pages solution, something like this is required:

error_page 400 401 402 403 ... 408 409 $scheme://$host:$server_port/errorpages/$status.html;

The reason why BOTH named locations AND explicit locations fail is, for example:

If a request is: "www.somesite.com/page.html" (valid URI)
Then (in the virtual server log) $request_uri contains "/page.html" and $uri contains "/page.html"

If a request is: "www.somesite.com" (empty URI)
Then (in the virtual server log) $request_uri contains "/" and $uri contains "/"

If a request is: "www.somesite.com/%%" (invalid URI)
Then (in the virtual server log) $request_uri contains "-" and $uri contains ""

NOTE THAT THE INVALID REQUEST RECEIVED FROM THE CLIENT DOES NOT CONTAIN THE "-" CHARACTER (so from where did it come???) thus this custom error page directive will FAIL...

error_page 400 /errorpages/400.html;

...because NGINX will be attempting to serve this URL/URI with a "-" in it...

www.somesite.com-/errorpages/400.html

Which is invalid because of the "-" character, so whether error_page points to a named or explicit LOCATION is thus irrelevant.

I disagree that this isn't an issue because processing is ALREADY occurring in the addressed virtual server ($host) so it shouldn't be necessary to have to explicitly specify the host, and so on. Respective errors are logged in the respective virtual server logs, NOT in the engine log, thus NGINX "knows" to which virtual server the invalid URI was addressed. Perhaps this wasn't clear from the beginning.

"error_page 400 /errorpages/400.html;" should be sufficient, and so should "error_page 400 @errorpages;" because the circumstance is error handling; the URI on the virtual server/host is what's invalid, not the virtual server/host name in the request.

Every error_page example I've been able to find, even on "official" pages, doesn't indicate that the full "$scheme://$host:$server_port$errorpageuri" location is required to serve custom error pages, so at the very least the official docs should be amended to reflect this, which is why I'm reopening this issue.

Anyway, my custom 400 error page (and others) is being served for all virtual hosts for every invalid URI I've thrown at them so far. Shouldn't have been necessary to go to such a length to get this working though.

[Co6aka@…]

I forgot add that the "internal" directive apparently doesn't work with explicit URLs.

[Maxim Dounin]

The "full location" as you call it is not required. Using full URL, starting with "​http://" or "​https://", instruct nginx to return a redirection instead of an error page, see ​docs. To handle requests where URI cannot be parsed, you can use normal error_page syntax with URI, for example:

error_page 400 /400.html;

If you have further question on how to configure nginx, please use ​mailing list instead.

[Co6aka@…]

"error_page 400 /400.html;"

...doesn't work when the test URL is...

​https://mysite.com/%%

...because NGINX's "hardcoded" 400 error page is displayed instead.

[Maxim Dounin]

You are wrong, the error page configured is returned as long as it is configured in the correct server block. As previously suggested, if you have further question on how to configure nginx, please use ​mailing list instead.