Opened 19 months ago

Closed 18 months ago

Last modified 18 months ago

#1003 closed defect (fixed)

Growing amount of active connections with http/2

Reported by: baurmatt@… Owned by:
Priority: minor Milestone:
Component: nginx-core Version: 1.9.x
Keywords: http2 Cc:
Sensitive: no
uname -a: Linux dummy.hostname.de 2.6.32-042stab104.1 #1 SMP Thu Sep 17 11:38:20 MSK 2015 x86_64 Intel(R) Xeon(R) CPU E5620 @ 2.40GHz GenuineIntel GNU/Linux
nginx -V: nginx version: nginx/1.9.12 built with OpenSSL 1.0.2f 28 Jan 2016 (running with OpenSSL 1.0.2h 3 May 2016) TLS SNI support enabled configure arguments: --prefix=/usr --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error_log --pid-path=/run/nginx.pid --lock-path=/run/lock/nginx.lock --with-cc-opt=-I/usr/include --with-ld-opt=-L/usr/lib64 --http-log-path=/var/log/nginx/access_log --http-client-body-temp-path=/var/lib/nginx/tmp/client --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --with-http_v2_module --with-ipv6 --with-pcre --with-http_geoip_module --with-http_realip_module --with-http_stub_status_module --with-http_realip_module --add-module=external_module/ngx_cache_purge-2.3 --with-http_ssl_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-mail_imap_module --without-mail_pop3_module --without-mail_smtp_module --user='nginx --group=nginx'

Description

We're running a couple of Nginx HTTP load balancer. Since upgrading to 1.9.x/1.11.x and enabling http/2, we're seeing a growing amount of active connections.

Some plattform details:

  • Nginx runs in a Virtuozzo container
  • Guest system is Gentoo or Ubuntu 14.04
  • Nginx version we're seeing this problem:
    • 1.9.12
    • 1.0.15
    • 1.10.0
    • 1.11.1

All of them have in common, that the problem appeared after enabling http/2.

Attachments (6)

Bildschirmfoto 2016-06-22 um 14.40.28.png (42.2 KB) - added by baurmatt@… 19 months ago.
Screenshot 1 - Monitoring System
Bildschirmfoto 2016-06-22 um 14.41.40.png (51.1 KB) - added by baurmatt@… 19 months ago.
Screenshot 2 - Monitoring System
Bildschirmfoto 2016-06-22 um 14.42.28.png (46.9 KB) - added by baurmatt@… 19 months ago.
Screenshot 3 - Monitoring System
Bildschirmfoto 2016-06-22 um 14.43.12.png (55.5 KB) - added by baurmatt@… 19 months ago.
Screenshot 4 - Monitoring System
http2_socket_leak.patch (3.1 KB) - added by vbart 19 months ago.
nginx_status-pinpoint=1469078442,1469769642.png (82.8 KB) - added by GreenReaper@… 18 months ago.
Nginx connection graph before and after 1.11.3

Download all attachments as: .zip

Change History (10)

Changed 19 months ago by baurmatt@…

Screenshot 1 - Monitoring System

Changed 19 months ago by baurmatt@…

Screenshot 2 - Monitoring System

Changed 19 months ago by baurmatt@…

Screenshot 3 - Monitoring System

Changed 19 months ago by baurmatt@…

Screenshot 4 - Monitoring System

comment:1 Changed 19 months ago by vbart

Could you provide configuration and a debug log?

Changed 19 months ago by vbart

comment:2 Changed 19 months ago by vbart

This problem is likely similar to the one that has been already discussed and resolved in the mailing-list:
http://mailman.nginx.org/pipermail/nginx/2016-June/051021.html

I've attached above the current version of patch to fix the issue. Please try it and let me know if it helps in your case.

Last edited 19 months ago by vbart (previous) (diff)

comment:3 Changed 18 months ago by vbart

  • Resolution set to fixed
  • Status changed from new to closed

I suppose it to be fixed by e78eca6bfaf0, b5d1c17181ca, and 72282dd5884e in nginx 1.11.3.

comment:4 Changed 18 months ago by GreenReaper@…

I can confirm that this appears to be resolved - which I'm very happy about, as for a year it had caused nginx to overflow its disk allocation by blocking eviction, and ultimately exhausted disk space.

Changed 18 months ago by GreenReaper@…

Nginx connection graph before and after 1.11.3

Note: See TracTickets for help on using tickets.