Missing error pages + deny + try_files results in full body being returned

[Kevin Fischer]

We've experienced a very weird behaviour today, which potentially is a security hazard to incorrectly setup nginx servers.

We've tried to deny access to one directory, but nginx would always end up returning a 404 status code along with the full body response of the resource that we tried to deny.

Following are the parts of our config that caused this bug:

server {
    try_files  $uri $uri/ /index.php?$args;

    ...

    error_page  403 /errorpage/403.html;
    error_page  404 /errorpage/404.html;

    ...

    location /internal/ {
        allow 127.0.0.0/24;
        deny all;
    }

    ...

    location = /index.php { ... }

The problem with our setup was that the error page directory and its contents were missing. nginx correctly denied access by returning a 403, which lead to it trying to load the 403 error page file. As this file did not exist, it fell back to the 404 error page, which also didn't exist, so in the internal request that tried to resolve the 404 error page, try_files seemingly redirected to index.php, as configured. The result was that instead of a 403 response with appropriate error page, nginx returned a 404 with the body that should not have been accessible.

The error log even showed an "access forbidden by rule" entry, even though nginx ended up delivering the forbidden content.

As soon as we've created the error page files, everything worked as expected.

I understand that this is partly a user fault, as we forgot to create the error page directory; but never the less it seems to us like this edge case behaviour is just plain incorrect and a potential security hazard.

[Maxim Dounin]

As far as I understand the above configuration, it's not nginx who returned your protected content to user, it's your /index.php script who did it.

Overall, there seems to be a number of problems with the configuration and the /index.php code:

• you instruct nginx to fallback to /index.php for anything;
• you wrote /index.php in such a way that it can return arbitrary files to users;
• in particular, you allow /index.php to access files in a protected location and return them to users;
• you've configured error pages in a such way that they cannot be found and so falls back to the /index.php.

Fixing any of the above will prevent the behaviour you are seeing. Not sure how nginx can help here though, other than discouraging such fragile configurations in education materials and examples.

[Kevin Fischer]

You seem to misunderstand the problem here. It doesn't matter that it's a PHP script that is being executed; it could be any file. The problem is the combination of a try_files fallback in combination with resolving the error pages, which did not exist. nginx will fall back to index.php in my example, as configured by try_files. But again: this could be any file, and that file would be delivered with a 404 response code, despite nginx being instructed to deny access.

A request timeline (which might not be correct 100%, as I don't know the nginx source code - this is just what I figured out from debugging this):
1) GET /internal/ => 403, via deny all
2) GET /errorpage/403.html => 404, because the file doesn't exist => try next, according to try_files:
3) GET /errorpage/403.html/ => 404 => try next, according to try_files:
4) GET /index.php => exists, but is returned with 404 status code

I understand why nginx is showing this behaviour, but I'd argue that this shouldn't happen when resolving for error pages. It doesn't make sense to fall back to the original request when trying to resolve for an error page.

[Maxim Dounin]

Well, nginx was instructed to deny access to files in /internal/, and it did so. It was also instructed (effectively, by error_page and try_files) to return /index.php as and error page, and it did so. As long as /index.php did not contain protected content, this is fine from security point of view.

Note well that try_files doesn't change the response code, so 403 with /index.php body will be returned. This is what your configuration says to do, and this is what happens.

[Kevin Fischer]

Well yes, you're absolutely right. It's just weird that nginx will reuse try_files when resolving the error_page, which in my eyes is a bug, an unexpected behaviour. I've been using nginx for years and I absolutely didn't know it would do that. Maybe the common pitfalls page should mention this combination of try_files and other sub-request-spawning directives like error_page, as try_files is heavily promoted there and on the web for configurations exactly like ours, which means that this can potentially affect lots of web servers.

Thanks for your time.

[Maxim Dounin]

The error_page directive takes the URI argument. The request will be redirected to the URI specified, and then the request will be processed much like any other request, including location matching, rewrite directives, try_files, index and so on. The ​documentation is pretty clear on this.

As for the pitfalls page you are referring to, it is a wiki page and contains only user contributed content. Feel free to suggests any edits you think are right.