Custom Query (2297 matches)

Hi,

we have a reverse proxy setup with 2 upstreams, the decision to which upstream to route a request is based on value of a specific header. Lets say we want to route requests to upstreams based on a value of the user-agent header

          upstream banana {
             server banana-server;
           }
           upstream apple {
             server apple-server;
           }

          map $http_user_agent $proxied_server {
            default apple;

            "~*Firefox*" apple;
            "~*Chrome*" banana;
          }
 
          server {
              listen 8080;
              server_name localhost;
              location / {
                    proxy_pass http://$proxied_server;
              }
          }

the request is sent to the right upstream IP by a reverse proxy, but instead of taking the DNS name of the server specified in the "upstream" the reverse proxy takes the name of upstream as a value for the host header. So instead of "banana-server" it sends just "banana".

Setup Client -[HTTP(S)]-> HAProxy -[PROXY]-> nginx -[HTTPS]-> Application

We noticed in our logs some strange errors like this:

2019/10/07 12:53:47 [info] 61#61: *2 client sent plain HTTP request to HTTPS port while reading client request headers, client: 172.23.186.8, server: _, request: "GET / HTTP/1.1", host: "example.org"

The problem with this error is not the message itself. The problem is that the client IP is not the real client IP. It‘s the IP of the proxy sitting ahead of nginx. The communication over PROXY protocol works fine, and realip is configured well. All our other logs (access + error) containing the correct client IP except for a few like this one.

It seems that if you use the PROXY protocol errors, which occure early during the request processing (e.g., NGX_HTTP_TO_HTTPS), don‘t get logged with the client IP but with the proxy IP. At this point in processing, we already have the $proxy_protocol_addr/connection->proxy_protocol_addr but ngx_http_log_error always uses connection->addr_text.

This only matters for a few errors e.g., NGX_HTTP_TO_HTTPS and cert issues. After these checks, the realip module jumps in and replaces connection->addr_text with connection->proxy_protocol_addr. So all errors after this got logged correctly with the client IP.

Writing connections counter is gradually grow up as shown on munin graph after switch to http/2 with nginx 1.15.7 and clear only after service nginx restart.

I think that #1650 and #1610 is not my case, becouse these thikets do not associated with switching to http/2 — after disabling http/2 in my case everything is OK. I do not have ModSecurity as in #1650, do not have stream {} in config file as noted in ticket:1610#comment:1, max_size for proxy_cache_path in not configured as in #1163, but in logs I got some open socket ... left in connection ... alerts on reload.