id summary reporter owner description type status priority milestone component version resolution keywords cc uname nginx_version 1484 Timeouts when proxying to Apache and using Keepalive Veroslav Cenov "Hello, I have the following setup: 1) Apache backend 2) Nginx proxy The request path goes like: Browser --> Nginx [proxy_pass] --> Apache In order to use KeepAlive between Nginx --> Apache, I'm doing the following: 1) Enable HTTP 1.1 , and clean the Connection header: ---- server { .... location / { .... .. proxy_set_header Connection """"; proxy_http_version 1.1; proxy_pass http://upstream_80; ... ..... } } 2) Add keepalive to my upstream ---- http{ .... upstream upstream_80 { keepalive 100; server 127.0.0.1:8080 max_fails=0 ; } .... } ------------------ The configuration above (+ enabling KeepAlive in Apache upstream), leads to succesfull usage of KeepAlive between Nginx and Apache. But this causes some very weird bug: some requests are being dropped and not even processed by Apache. I'm doing benchmarks with the following: root@localhost# echo ss > test_post root@localhost# /usr/local/apache/bin/ab -k -p test_post -c 100 -n 2000000 http://mydomain.com/non_existing_file.html Where ""non_existing_file.html"" is non-existing file, which returns 404 , but the request itself makes it through Apache. (It doesn't matter if file exists or not, I'm using non-existent, because the test goes faster) For a 2 000 000 hits benchmark I'm receiving bunch of timeouts , logged in Nginx error log and looking like this: ''----- 2018/02/16 19:04:37 [crit] 17155#17155: *231641 connect() to 127.0.0.1:8080 failed (99: Cannot assign requested address) while connecting to upstream, client: 127.0.0.1, server: static.superhosting.bg, request: ""POST /ss.html HTTP/1.0"", upstream: ""http://127.0.0.1:8080/ss.html"", host: ""static.superhosting.bg"" 2018/02/16 19:04:37 [crit] 17152#17152: *231633 connect() to 127.0.0.1:8080 failed (99: Cannot assign requested address) while connecting to upstream, client: 127.0.0.1, server: static.superhosting.bg, request: ""POST /ss.html HTTP/1.0"", upstream: ""http://127.0.0.1:8080/ss.html"", host: ""static.superhosting.bg"" 2018/02/16 19:04:37 [crit] 17152#17152: *232237 connect() to 127.0.0.1:8080 failed (99: Cannot assign requested address) while connecting to upstream, client: 127.0.0.1, server: static.superhosting.bg, request: ""POST /ss.html HTTP/1.0"", upstream: ""http://127.0.0.1:8080/ss.html"", host: ""static.superhosting.bg"" 2018/02/16 19:04:37 [crit] 17152#17152: *231387 connect() to 127.0.0.1:8080 failed (99: Cannot assign requested address) while connecting to upstream, client: 127.0.0.1, server: static.superhosting.bg, request: ""POST /ss.html HTTP/1.0"", upstream: ""http://127.0.0.1:8080/ss.html"", host: ""static.superhosting.bg"" 2018/02/16 19:04:37 [crit] 17152#17152: *232756 connect() to 127.0.0.1:8080 failed (99: Cannot assign requested address) while connecting to upstream, client: 127.0.0.1, server: static.superhosting.bg, request: ""POST /ss.html HTTP/1.0"", upstream: ""http://127.0.0.1:8080/ss.html"", host: ""static.superhosting.bg"" ------'' Apache itself doesn't log anything (even with LogLevel +trace9). If I disable the ""keepalive"" directive inside upstream {} (nginx.conf), the problem becomes immediately fixed. I was able to successfully simulate this behavior to the following Nginx versions: Nginx - 1.12.2 , 1.11.8 , 1.13.7 Also this behaviour is observed over both http/https connections. " defect closed major other 1.12.x duplicate keepalive upstream timeout veroslav.cenov@… Linux elastic 3.10.0-693.17.1.el7.x86_64 #1 SMP Thu Jan 25 20:13:58 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux "nginx version: nginx/1.12.2 built by gcc 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) built with OpenSSL 1.0.2k-fips 26 Jan 2017 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie' "