Opened 10 years ago

Closed 9 years ago

Last modified 9 years ago

#1003 closed defect (fixed)

Growing amount of active connections with http/2

Reported by: Matthias Baur Owned by:
Priority: minor Milestone:
Component: nginx-core Version: 1.9.x
Keywords: http2 Cc:
uname -a: Linux dummy.hostname.de 2.6.32-042stab104.1 #1 SMP Thu Sep 17 11:38:20 MSK 2015 x86_64 Intel(R) Xeon(R) CPU E5620 @ 2.40GHz GenuineIntel GNU/Linux
nginx -V: nginx version: nginx/1.9.12
built with OpenSSL 1.0.2f 28 Jan 2016 (running with OpenSSL 1.0.2h 3 May 2016)
TLS SNI support enabled
configure arguments: --prefix=/usr --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error_log --pid-path=/run/nginx.pid --lock-path=/run/lock/nginx.lock --with-cc-opt=-I/usr/include --with-ld-opt=-L/usr/lib64 --http-log-path=/var/log/nginx/access_log --http-client-body-temp-path=/var/lib/nginx/tmp/client --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --with-http_v2_module --with-ipv6 --with-pcre --with-http_geoip_module --with-http_realip_module --with-http_stub_status_module --with-http_realip_module --add-module=external_module/ngx_cache_purge-2.3 --with-http_ssl_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-mail_imap_module --without-mail_pop3_module --without-mail_smtp_module --user='nginx --group=nginx'

Description

We're running a couple of Nginx HTTP load balancer. Since upgrading to 1.9.x/1.11.x and enabling http/2, we're seeing a growing amount of active connections.

Some plattform details:

  • Nginx runs in a Virtuozzo container
  • Guest system is Gentoo or Ubuntu 14.04
  • Nginx version we're seeing this problem:
    • 1.9.12
    • 1.0.15
    • 1.10.0
    • 1.11.1

All of them have in common, that the problem appeared after enabling http/2.

Attachments (6)

Bildschirmfoto 2016-06-22 um 14.40.28.png (42.2 KB ) - added by Matthias Baur 10 years ago.
Screenshot 1 - Monitoring System
Bildschirmfoto 2016-06-22 um 14.41.40.png (51.1 KB ) - added by Matthias Baur 10 years ago.
Screenshot 2 - Monitoring System
Bildschirmfoto 2016-06-22 um 14.42.28.png (46.9 KB ) - added by Matthias Baur 10 years ago.
Screenshot 3 - Monitoring System
Bildschirmfoto 2016-06-22 um 14.43.12.png (55.5 KB ) - added by Matthias Baur 10 years ago.
Screenshot 4 - Monitoring System
http2_socket_leak.patch (3.1 KB ) - added by Valentin V. Bartenev 10 years ago.
nginx_status-pinpoint=1469078442,1469769642.png (82.8 KB ) - added by Laurence 'GreenReaper' Parry 9 years ago.
Nginx connection graph before and after 1.11.3

Download all attachments as: .zip

Change History (10)

by Matthias Baur, 10 years ago

Screenshot 1 - Monitoring System

by Matthias Baur, 10 years ago

Screenshot 2 - Monitoring System

by Matthias Baur, 10 years ago

Screenshot 3 - Monitoring System

by Matthias Baur, 10 years ago

Screenshot 4 - Monitoring System

comment:1 by Valentin V. Bartenev, 10 years ago

Could you provide configuration and a debug log?

by Valentin V. Bartenev, 10 years ago

Attachment: http2_socket_leak.patch added

comment:2 by Valentin V. Bartenev, 10 years ago

This problem is likely similar to the one that has been already discussed and resolved in the mailing-list:
http://mailman.nginx.org/pipermail/nginx/2016-June/051021.html

I've attached above the current version of patch to fix the issue. Please try it and let me know if it helps in your case.

Last edited 10 years ago by Valentin V. Bartenev (previous) (diff)

comment:3 by Valentin V. Bartenev, 9 years ago

Resolution: fixed
Status: newclosed

I suppose it to be fixed by e78eca6bfaf0, b5d1c17181ca, and 72282dd5884e in nginx 1.11.3.

comment:4 by Laurence 'GreenReaper' Parry, 9 years ago

I can confirm that this appears to be resolved - which I'm very happy about, as for a year it had caused nginx to overflow its disk allocation by blocking eviction, and ultimately exhausted disk space.

by Laurence 'GreenReaper' Parry, 9 years ago

Nginx connection graph before and after 1.11.3

Note: See TracTickets for help on using tickets.