Reloading config with differently cased proxy_set_header field name does not change field name in proxied request

[Oliver Friedrich]

Today, I was pulling my hair out because I wanted to change my wrong X-Forwarded-For configuration from

proxy_set_header    X_FORWARDED_FOR    $proxy_add_x_forwarded_for;

to the correct version

proxy_set_header    X-Forwarded-For    $proxy_add_x_forwarded_for;

I tried reloading the config using service nginx relaod a few times but without success.

The solution that finally worked was to comment the above line completely, reload the configuration, uncomment the corrected version and reload again. Now, the rewritten requests contain the X-Forwarded-For header instead of the wrong X_FORWARDED_FOR one.

Since I don't know nginx's source I cannot tell what scope this problem has - it might be a larger problem, altogether.

[Maxim Dounin]

There are multiple reasons why a configuration reload can fail, and many of them transient. Try looking into error log to find out what happened in your particular case. Note well that "relaod" isn't correct, "reload" is. Note also that nginx 1.5.2 is long time obsolete, current version is 1.7.1.

[Oliver Friedrich]

Thanks, Maxim, you were of course right. I work with NGINX only rarely and checking the error.log file I found out that I had been missing a semicolon all the time so the configuration was never actually applied.

The response by service nginx reload is misleading in this case: nginx neu laden: [ OK ] (on a German system). It would help if the status message indicated that the configuration is erronous (but this maybe fixed in newer versions, so I'll update asap).

[Maxim Dounin]

The service command isn't part of nginx but rather your OS (or, rather, it calls a script provided by an nginx package you use), and it merely sends the ​HUP signal to nginx to initiate a configuration reload. Waiting for actual configuration reload results is certainly out of scope of this action though. Moreover, such waiting can't be easily automated as actual reload of a configuration may take very long time, especially including graceful shutdown of old worker processes. And in many cases there is no need to wait for previous reload to complete before doing another one.

To check if a configuration is correct, consider using nginx -t. Many control scripts provide a configtest target for this.

[Oliver Friedrich]

Even though I was aware that service is part of the OS, I wasn't aware of the fact that it would not return the underlying service's response. Thanks for clearing this up and for the hint to nginx -t.

To reload nginx, nginx -s reload will be my choice from now on.

[Maxim Dounin]

The nginx -s reload is a wrong choice. The only difference is extra parsing of a configuration, and a pid file used from the configuration. Both actions may be incorrect in some cases (e.g., if you are in the middle of an upgrade, and nginx binary on disk differs from one in memory; or you've changed pid file location) and still doesn't cover various errors not related to a configuration parsing (e.g., a configuration reload can fail due to a listen socket being busy). The -s option exists mainly for win32 hosts, where there are no easy way to send signals.

The only correct approach is to look into logs and/or processes to ensure the reload went fine.

[Oliver Friedrich]

Thanks for clearing this up. So I understand correctly that service nginx reload should be used for configuration reload, possibly after verifying with nginx -t that the configuration is valid?

I appreciate your patience and help with understanding nginx better!

[Maxim Dounin]

Yes. And after doing a configuration reload, you should look into logs to make sure a reload went fine.

[Oliver Friedrich]

Doing that every time now. Thanks.