Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#1003 closed defect (fixed)

Growing amount of active connections with http/2

Reported by: Matthias Baur Owned by:
Priority: minor Milestone:
Component: nginx-core Version: 1.9.x
Keywords: http2 Cc:
uname -a: Linux dummy.hostname.de 2.6.32-042stab104.1 #1 SMP Thu Sep 17 11:38:20 MSK 2015 x86_64 Intel(R) Xeon(R) CPU E5620 @ 2.40GHz GenuineIntel GNU/Linux
nginx -V: nginx version: nginx/1.9.12
built with OpenSSL 1.0.2f 28 Jan 2016 (running with OpenSSL 1.0.2h 3 May 2016)
TLS SNI support enabled
configure arguments: --prefix=/usr --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error_log --pid-path=/run/nginx.pid --lock-path=/run/lock/nginx.lock --with-cc-opt=-I/usr/include --with-ld-opt=-L/usr/lib64 --http-log-path=/var/log/nginx/access_log --http-client-body-temp-path=/var/lib/nginx/tmp/client --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --with-http_v2_module --with-ipv6 --with-pcre --with-http_geoip_module --with-http_realip_module --with-http_stub_status_module --with-http_realip_module --add-module=external_module/ngx_cache_purge-2.3 --with-http_ssl_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-mail_imap_module --without-mail_pop3_module --without-mail_smtp_module --user='nginx --group=nginx'

Description

We're running a couple of Nginx HTTP load balancer. Since upgrading to 1.9.x/1.11.x and enabling http/2, we're seeing a growing amount of active connections.

Some plattform details:

  • Nginx runs in a Virtuozzo container
  • Guest system is Gentoo or Ubuntu 14.04
  • Nginx version we're seeing this problem:
    • 1.9.12
    • 1.0.15
    • 1.10.0
    • 1.11.1

All of them have in common, that the problem appeared after enabling http/2.

Attachments (6)

Bildschirmfoto 2016-06-22 um 14.40.28.png (42.2 KB ) - added by Matthias Baur 8 years ago.
Screenshot 1 - Monitoring System
Bildschirmfoto 2016-06-22 um 14.41.40.png (51.1 KB ) - added by Matthias Baur 8 years ago.
Screenshot 2 - Monitoring System
Bildschirmfoto 2016-06-22 um 14.42.28.png (46.9 KB ) - added by Matthias Baur 8 years ago.
Screenshot 3 - Monitoring System
Bildschirmfoto 2016-06-22 um 14.43.12.png (55.5 KB ) - added by Matthias Baur 8 years ago.
Screenshot 4 - Monitoring System
http2_socket_leak.patch (3.1 KB ) - added by Valentin V. Bartenev 8 years ago.
nginx_status-pinpoint=1469078442,1469769642.png (82.8 KB ) - added by Laurence 'GreenReaper' Parry 8 years ago.
Nginx connection graph before and after 1.11.3

Download all attachments as: .zip

Change History (10)

by Matthias Baur, 8 years ago

Screenshot 1 - Monitoring System

by Matthias Baur, 8 years ago

Screenshot 2 - Monitoring System

by Matthias Baur, 8 years ago

Screenshot 3 - Monitoring System

by Matthias Baur, 8 years ago

Screenshot 4 - Monitoring System

comment:1 by Valentin V. Bartenev, 8 years ago

Could you provide configuration and a debug log?

by Valentin V. Bartenev, 8 years ago

Attachment: http2_socket_leak.patch added

comment:2 by Valentin V. Bartenev, 8 years ago

This problem is likely similar to one that has been already discussed and resolved in the mailing-list:
http://mailman.nginx.org/pipermail/nginx/2016-June/051021.html

I've attached above the current version of patch to fix the issue. Please try it and let me know if it helps in your case.

Version 0, edited 8 years ago by Valentin V. Bartenev (next)

comment:3 by Valentin V. Bartenev, 8 years ago

Resolution: fixed
Status: newclosed

I suppose it to be fixed by e78eca6bfaf0, b5d1c17181ca, and 72282dd5884e in nginx 1.11.3.

comment:4 by Laurence 'GreenReaper' Parry, 8 years ago

I can confirm that this appears to be resolved - which I'm very happy about, as for a year it had caused nginx to overflow its disk allocation by blocking eviction, and ultimately exhausted disk space.

by Laurence 'GreenReaper' Parry, 8 years ago

Nginx connection graph before and after 1.11.3

Note: See TracTickets for help on using tickets.