Opened 3 years ago
Closed 3 years ago
#2192 closed defect (duplicate)
Upstream Keepalive - TCP connection to the upstream closes
| Reported by: | https://stackoverflow.com/users/10634976/patrick-scott-best | Owned by: | |
|---|---|---|---|
| Priority: | minor | Milestone: | |
| Component: | nginx-core | Version: | 1.19.x |
| Keywords: | upstream keepalive | Cc: | |
| uname -a: | Linux srv001b-media-01 5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 x86_64 GNU/Linux | ||
| nginx -V: |
nginx version: nginx/1.20.0
built by gcc 8.3.0 (Debian 8.3.0-6) built with OpenSSL 1.1.1d 10 Sep 2019 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.20.0/debian/debuild-base/nginx-1.20.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie' |
||
Description
I have an NGINX upstream definition that involves rotation through several servers to find content. The content requested is only on one of these servers, so I keep "trying" until I find the content, or die after I've exhausted all upstreams. This works well, however, there is a delay in "fishing through" any upstream servers that don't have the content to find the correct one due to TCP overhead.
The server on which the content is found has successfully established a connection using the keepalive / HTTP 1.1 method.
The problem I have is that NGINX closes the TCP connection to any upstream server that returns a 404. Thus, the next time around, a TCP connection must be established before retrieving another 404 and moving on to (the correct) webserver.
When a connection is established with the correct upstream server that bears 200-fruit, that upstream connection is successfully established with keepalive, and subsequent attempts at the content are performed through this established connection as expected. This is good - packet capture and netstats show the longevity of this connection for multiple subsequent HTTP requests.
My problem is that I would like to more rapidly run through the list of (bad) servers, and not have to wait for another TCP connection to be brought up each and every "try".
The Downstream NGINX server is the one that initiates the closure of the TCP connection with FIN ACK, not the upstream.
Downstream Config:
proxy_socket_keepalive on;
upstream myupstream {
server 10.x.x.x:32700 max_fails=0;
server 10.y.y.y:32700 max_fails=0;
server 10.z.z.z:32700 max_fails=0;
keepalive 32;
}
location myfile.jpg {
proxy_set_header Connection "";
proxy_http_version 1.1;
proxy_intercept_errors on;
proxy_pass http://myupstream;
proxy_next_upstream error timeout http_500 http_502 http_503 http_404;
proxy_next_upstream_tries 3;
proxy_read_timeout 2s;
}
Even the upstream server desires to keep this connection open, but NGINX downstream terminates the TCP connection regardless.
HTTP/1.1 404 Not Found
Server: nginx
Date: Thu, 20 May 2021 22:44:20 GMT
Content-Type: text/html
Content-Length: 146
Connection: keep-alive
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
Attachments (1)
Change History (3)
by , 3 years ago
| Attachment: | Upstream Keepalive.png added |
|---|
comment:1 by , 3 years ago
Similarly to error interception with proxy_intercept_errors (see ticket #2033), moving to the next upstream server via proxy_next_upstream implies that nginx cannot maintain keepalive connection if the response body is not empty, since it does not try to read the body.
Closing this as a duplicate of #2033.
comment:2 by , 3 years ago
| Resolution: | → duplicate |
|---|---|
| Status: | new → closed |
Wireshark clip that demonstrates the downstream is terminating the TCP connection.