set_real_ip_from not working in 1.10.x

[exobuzz@…]

nginx version:

Linux 4.7.0-0.bpo.1-amd64 #1 SMP Debian 4.7.8-1~bpo8+1 (2016-10-19) x86_64 GNU/Linux

Upgraded from 1.9.10 to 1.10.2 and set_real_ip_from is no longer working ( running behind varnish)

Other users are having the same issue - ​https://github.com/gplessis/dotdeb-nginx/issues/87 (Using dotdeb packages)

real_ip nginx config

set_real_ip_from IP_OF_SERVER;
real_ip_header X-Forwarded-For;
real_ip_recursive on;

This works fine on 1.9.10 and in the logs are the ip of the client. However when using 1.10.2 the IP_OF_SERVER is shown instead

setting the log format to

        log_format timed_combined '$remote_addr - $remote_user [$time_local] '
            '"$request" $status $body_bytes_sent '
            '"$http_referer" "$http_user_agent" '
            'http_x_forwarded_for = $http_x_forwarded_for,
real_ip_remote_addr = $realip_remote_addr';

gives:

IP_OF_SERVER - - [14/Nov/2016:17:49:16 +0000] "GET /nginx_test HTTP/1.1" 404 9553 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0" http_x_forwarded_for = IP_OF_CLIENT, IP_OF_CLIENT, real_ip_remote_addr = IP_OF_SERVER

IP_OF_SERVER = IP of the webserver
IP_OF_CLIENT = IP of my client which should be used for $remote_addr

switching back to 1.9.10 and all is fine again with IP_OF_CLIENT showing in the log for $remote_addr

[Maxim Dounin]

Just tested, and it works fine here.

Tested with the following configuration:

http {
    log_format timed_combined '$remote_addr - $remote_user [$time_local] '
            '"$request" $status $body_bytes_sent '
            '"$http_referer" "$http_user_agent" '
            'http_x_forwarded_for = $http_x_forwarded_for, '
            'real_ip_remote_addr = $realip_remote_addr';

    access_log /dev/stderr timed_combined;

    server {
        listen       8080;

        set_real_ip_from 127.0.0.1;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;

        location / {
            return 200 "addr:$remote_addr\n";
        }
    }
}

Request / response:

GET / HTTP/1.0
X-Forwarded-For: 127.0.0.2, 127.0.0.2

HTTP/1.1 200 OK
Server: nginx/1.10.2
Date: Tue, 15 Nov 2016 16:14:49 GMT
Content-Type: text/plain
Content-Length: 15
Connection: close

addr:127.0.0.2

Access log with the format in question:

127.0.0.2 - - [15/Nov/2016:19:14:49 +0300] "GET / HTTP/1.0" 200 15 "-" "-" http_x_forwarded_for = 127.0.0.2, 127.0.0.2, real_ip_remote_addr = 127.0.0.1

Please provide full configuration to reproduce the problem, and a ​debug log with a request where realip does not work correctly.

Please avoid any modifications to the data provided, as data modifications can easily hide the root cause of the problem you see. If you can't disclose some information like IP addresses, please reproduce the problem in a sandbox with private / local IP addresses instead of modifying the data.

[exobuzz@…]

This might help - From another user from ​https://github.com/gplessis/dotdeb-nginx/issues/87#issuecomment-260696647

Based on the results on a Gentoo linux server I ran some more tests on a debian server running nginx v1.10.2. It seems like set_real_ip_from is working fine as long as I'm not using $remote_addr any further in the config.

With this config in use, realip works as expected and the X-Forward-For content shows up in the log files as the source IP (detail and crit both use $remote_addr):

server {
    listen       81;
    server_name   _;
    access_log   /var/log/nginx/access.log detail;
    error_log    /var/log/nginx/error.log crit;
    root         /var/www

    set_real_ip_from    0.0.0.0/0;
    real_ip_header     X-Forwarded-For;

    location / {
    }
}

This config seems to break realip functionality:

server {
    listen       81;
    server_name   _;
    access_log   /var/log/nginx/access.log detail;
    error_log    /var/log/nginx/error.log crit;
    root         /var/www

    set_real_ip_from    0.0.0.0/0;
    real_ip_header     X-Forwarded-For;

    set $test123 $remote_addr;

    location / {
    }
}

[Maxim Dounin]

Both configurations provided work fine as well.

Using $remote_addr in various parts of the configuration can affect realip operation though. If the $remote_addr variable is used _before_ realip modified the address, original address will be returned, and the value of the variable will be cached till the end of the request. This can be observed in a configuration like this:

server {
    listen       8080;

    set $test123 $remote_addr;

    location / {
        set_real_ip_from 127.0.0.1;
        real_ip_header X-Forwarded-For;
        real_ip_recursive on;
    }
}

Note that realip module is configured in a particular location, and thus it can only change the address once the location is selected. On the other hand, the $remote_addr variable is used before selecting a location. The value obtained on the first access to the variable (that is, unchanged one) will be cached and returned on subsequent accesses.

Log with the above configuration and the same request will be as follows:

127.0.0.1 - - [15/Nov/2016:22:46:18 +0300] "GET / HTTP/1.0" 200 612 "-" "-" http_x_forwarded_for = 127.0.0.2, 127.0.0.2, real_ip_remote_addr = 127.0.0.1

That is, cached $remote_addr value is logged. This behaviour is universal and do not depend on nginx version.

Solution is to avoid configurations where you try to access $remote_addr before it is changed by the realip module. Most trivial way to do so is to configure set_real_ip_from as server level instead of location one.

Closing this as duplicate of #603, which is about realip and $remote_addr caching.

[exobuzz@…]

Looks like it is a problem specific to the dotdeb packages so will continue there. Thanks for the feedback.