Problem with proxy_pass when using upstream and map

[dimvvv@…]

I have a problem with proxy_pass directive when using upstream name wrapped into variable via map.

This is my nginx.conf:

user              nginx;
worker_processes  2;
worker_rlimit_nofile 30000;

error_log  /var/log/nginx/error.log;

pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    client_max_body_size 50m;
    sendfile        on;
    keepalive_timeout  65;

    #include /etc/nginx/conf.d/*.conf;

    map $server_addr $backend {
        "default"                  "backend1";
        "~^(172\.22\.33.*)"        "backend1";
        "~^(172\.20\.16.*)"        "backend2";
    }

    upstream backend1 {
        server 172.22.33.49;
        server 172.22.33.69 backup;
    }

    upstream backend2 {
        server 172.22.33.69;
        server 172.22.33.49 backup;
    }

    server {
        listen       172.22.33.112:80;
        server_name  test.s7.aero;

        access_log  /var/log/nginx/test.access.log;
        error_log  /var/log/nginx/test.error.log;

        location /path1/ {
            proxy_pass http://backend1/simple/;
        }

        location /path2/ {
            proxy_pass http://backend2/simple/;
        }

        location /map/ {
            proxy_pass http://$backend/simple/;
        }
    }

}

As you can see, I have a two backend servers, and two upstreams. I want to use different upstreams (different order of backends) depending on some external condition (map directive result). If I'm tryng proxy_pass to upstreams separately, it's ok. But if I try proxy_pass to $backend all part of url after location will be lost.

Will show in practice.

Request to first backend directly:

curl http://172.22.33.49/simple/test.html -v
*   Trying 172.22.33.49...
* Connected to 172.22.33.49 (172.22.33.49) port 80 (#0)
> GET /simple/test.html HTTP/1.1
> Host: 172.22.33.49
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx/1.8.0
< Date: Thu, 01 Jun 2017 04:23:21 GMT
< Content-Type: text/html
< Content-Length: 18
< Last-Modified: Thu, 01 Jun 2017 04:22:50 GMT
< Connection: keep-alive
< ETag: "592f969a-12"
< Accept-Ranges: bytes
< 
I'm first backend
* Connection #0 to host 172.22.33.49 left intact

Request to second backend directly:

curl http://172.22.33.69/simple/test.html -v
*   Trying 172.22.33.69...
* Connected to 172.22.33.69 (172.22.33.69) port 80 (#0)
> GET /simple/test.html HTTP/1.1
> Host: 172.22.33.69
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx/1.8.0
< Date: Thu, 01 Jun 2017 04:23:41 GMT
< Content-Type: text/html
< Content-Length: 19
< Last-Modified: Thu, 01 Jun 2017 04:23:15 GMT
< Connection: keep-alive
< ETag: "592f96b3-13"
< Accept-Ranges: bytes
< 
I'm second backend
* Connection #0 to host 172.22.33.69 left intact

Request to first upstream:

curl http://test.s7.aero/path1/test.html -v
*   Trying 172.22.33.112...
* Connected to test.s7.aero (172.22.33.112) port 80 (#0)
> GET /path1/test.html HTTP/1.1
> Host: test.s7.aero
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx/1.12.0
< Date: Thu, 01 Jun 2017 04:24:23 GMT
< Content-Type: text/html
< Content-Length: 18
< Connection: keep-alive
< Last-Modified: Thu, 01 Jun 2017 04:22:50 GMT
< ETag: "592f969a-12"
< Accept-Ranges: bytes
< 
I'm first backend
* Connection #0 to host test.s7.aero left intact

Request to second upstream:

curl http://test.s7.aero/path2/test.html -v
*   Trying 172.22.33.112...
* Connected to test.s7.aero (172.22.33.112) port 80 (#0)
> GET /path2/test.html HTTP/1.1
> Host: test.s7.aero
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx/1.12.0
< Date: Thu, 01 Jun 2017 04:25:09 GMT
< Content-Type: text/html
< Content-Length: 19
< Connection: keep-alive
< Last-Modified: Thu, 01 Jun 2017 04:23:15 GMT
< ETag: "592f96b3-13"
< Accept-Ranges: bytes
< 
I'm second backend
* Connection #0 to host test.s7.aero left intact

And now trying to request via $backend:

curl http://test.s7.aero/map/test.html -v
*   Trying 172.22.33.112...
* Connected to test.s7.aero (172.22.33.112) port 80 (#0)
> GET /map/test.html HTTP/1.1
> Host: test.s7.aero
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 403 Forbidden
< Server: nginx/1.12.0
< Date: Thu, 01 Jun 2017 04:25:48 GMT
< Content-Type: text/html
< Content-Length: 168
< Connection: keep-alive
< 
<html>
<head><title>403 Forbidden</title></head>
<body bgcolor="white">
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.8.0</center>
</body>
</html>
* Connection #0 to host test.s7.aero left intact

In the backend logs I see an error:

172.22.33.112 - - [01/Jun/2017:11:25:48 +0700] "GET /simple/ HTTP/1.0" 403 168 "-" "curl/7.47.0" "-"

The problem is all data (including arguments) after /map/ will be lost.
Even if I try this:

curl http://test.s7.aero/map/a/b/d/e/f/g/h/j/k/test.html -v
*   Trying 172.22.33.112...
* Connected to test.s7.aero (172.22.33.112) port 80 (#0)
> GET /map/a/b/d/e/f/g/h/j/k/test.html HTTP/1.1
> Host: test.s7.aero
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 403 Forbidden
< Server: nginx/1.12.0
< Date: Thu, 01 Jun 2017 04:31:05 GMT
< Content-Type: text/html
< Content-Length: 168
< Connection: keep-alive
< 
<html>
<head><title>403 Forbidden</title></head>
<body bgcolor="white">
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.8.0</center>
</body>
</html>
* Connection #0 to host test.s7.aero left intact

In backend logs URI the same:

172.22.33.112 - - [01/Jun/2017:11:31:05 +0700] "GET /simple/ HTTP/1.0" 403 168 "-" "curl/7.47.0" "-"

Summary: I expect the same behavior when using in proxy_pass upstream name and variable containing upstream name. But in practice first method working properly and second method not.

P.S. 403 on /simple/ is not a problem. Direct request show the same result:
P.P.S Sorry for the wrong nginx version, there isn't actual stable version in your dropdown list

curl http://172.22.33.49/simple/ -v
*   Trying 172.22.33.49...
* Connected to 172.22.33.49 (172.22.33.49) port 80 (#0)
> GET /simple/ HTTP/1.1
> Host: 172.22.33.49
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 403 Forbidden
< Server: nginx/1.8.0
< Date: Thu, 01 Jun 2017 04:42:30 GMT
< Content-Type: text/html
< Content-Length: 168
< Connection: keep-alive
< 
<html>
<head><title>403 Forbidden</title></head>
<body bgcolor="white">
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.8.0</center>
</body>
</html>
* Connection #0 to host 172.22.33.49 left intact

curl http://172.22.33.69/simple/ -v
*   Trying 172.22.33.69...
* Connected to 172.22.33.69 (172.22.33.69) port 80 (#0)
> GET /simple/ HTTP/1.1
> Host: 172.22.33.69
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 403 Forbidden
< Server: nginx/1.8.0
< Date: Thu, 01 Jun 2017 04:42:55 GMT
< Content-Type: text/html
< Content-Length: 168
< Connection: keep-alive
< 
<html>
<head><title>403 Forbidden</title></head>
<body bgcolor="white">
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.8.0</center>
</body>
</html>
* Connection #0 to host 172.22.33.69 left intact

[Maxim Dounin]

Quoting documentation, as updated several days ago per ticket #803:

When variables are used in proxy_pass:

location /name/ {
    proxy_pass http://127.0.0.1$request_uri;
}

In this case, if URI is specified in the directive, it is passed to the server as is, replacing the original request URI.

See ​http://nginx.org/r/proxy_pass for more information.