Opened 6 years ago
Last modified 2 years ago
#1579 new defect
Mirror subrequests ignore the keepalive flag
| Reported by: | Owned by: | ||
|---|---|---|---|
| Priority: | minor | Milestone: | |
| Component: | other | Version: | 1.14.x |
| Keywords: | Cc: | ||
| uname -a: | Linux z-space 4.13.0-43-generic #48~16.04.1-Ubuntu SMP Thu May 17 12:56:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | ||
| nginx -V: |
nginx version: nginx/1.14.0
built by gcc 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.4) built with OpenSSL 1.0.1f 6 Jan 2014 (running with OpenSSL 1.0.2g 1 Mar 2016) TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie' |
||
Description
I'm attempting to use the mirror directive to soak test a new backend service, and am experiencing issues due to the upstream connections being closed after each mirror subrequest. This makes the test impossible, as we hit port exhaustion in < 60s.
I've built a test case using a simple Go server and ab for load testing, attached as test-site.conf and run-test.sh. This test case demonstrates that keepalive functionality works fine for the upstream, except in the case of mirror (and auth_request) subrequests.
I've also done some source diving, and discovered that this can be worked around by changing ngx_http_mirror_module.c:171 to set sr->discard_body = 1 instead of sr->header_only = 1, though I have no idea what side effects this might cause.
Attachments (2)
Change History (5)
by , 6 years ago
| Attachment: | test-site.conf added |
|---|
comment:1 by , 6 years ago
A mirror subrequest response body is useless both for client output and for saving in memory. That's why the header_only flag is set for a mirror subrequest. The flag triggers some optimizations which may disable upstream keepalive as a side-effect.
This is not the only case when upstream keepalive gets disabled as a result of optimization. Another case is when upstream response comes before request body is fully sent. In most cases this works fine since establishing a new connection is usually faster than trying to send or read useless data. But probably we need a directive to force upstream connection consistency (send all and read all), which will enable keepalive for special cases. Currently we have no specific plans about this issue.
A simple and dirty workaround for your case is enabling proxy_cache or proxy_store.
comment:2 by , 5 years ago
Actually, I've encountered this issue, and was able to bypass the auth keep alive disable issue, by using the NGINX JavaScript. See this post https://runkiss.blogspot.com/2020/03/nginx-performance-tuning.html for full details
comment:3 by , 2 years ago
As this is also affecting auth and there are multiple workarounds (e.g. https://github.com/kubernetes/ingress-nginx/pull/8219) is there any chance to bump the priority of this?
Test site config