#1509 closed defect (invalid)

NGINX with Stream utilizing memory exponentially on each connection

Reported by: nikhilvs@… Owned by:
Priority: critical Milestone:
Component: other Version: 1.12.x
Keywords: memory leak, stream Cc:
uname -a: Linux ip-10-0-16-72.ec2.internal 4.9.85-46.56.amzn2.x86_64 #1 SMP Wed Mar 7 02:05:10 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.12.2 built by gcc 7.2.1 20170915 (Red Hat 7.2.1-2) (GCC) built with OpenSSL 1.0.2k-fips 26 Jan 2017 TLS SNI support enabled configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'

Description

Is it normal that NGINX uses extra memory on each connection ?

We are using Nginx to terminate TLS and upstream to Main Application.
For which we are using Stream configuration as protocol is not HTTP.

What we observe is :
with 100000 worker connections on start Nginx worker process uses 44KB and increases to 280MB for 6286 client connection.(I have seen it going to 1.4 GB for 35K concurrent connections)
Total connections including upstream : 12528
i.e netstat -natp|grep nginx|wc -l
12686

UID PID PPID C SZ RSS PSR STIME TTY TIME CMD
root 2568 1 0 15437 5860 1 Mar15 ? 00:00:00 nginx: master process /usr/sbin/nginx
nginx 3449 2568 0 98207 295576 1 Mar19 ? 00:04:53 nginx: worker process

  • Am i missing some configuration ?
  • Or Is this normal ?
  • Is this some kind of memory leak ?

Looking forward some solution, i have attached Nginx config file along with ticket.
Thanks,
Nikhil

Attachments (1)

nginx-1.conf (1.2 KB) - added by nikhilvs@… 21 months ago.
Conf file

Download all attachments as: .zip

Change History (2)

Changed 21 months ago by nikhilvs@…

Conf file

comment:1 Changed 21 months ago by mdounin

  • Resolution set to invalid
  • Status changed from new to closed

For each established connection in the stream module nginx uses up to two buffers of size set with proxy_buffer_size, and some minor overhead for various state structures. That is, up to 32k per connection with default settings.

Additionally, using SSL implies SSL-specific overhead which depends on the OpenSSL library used and various options, and a special 16k buffer used to coalesce different response parts.

In the memory usage numbers you've provided memory usage per connection is about 45k. This agrees with the expected memory usage in your configuration, there are no reasons to assume there are any leaks.

If you have further questions, please use support options available. Thank you.

Note: See TracTickets for help on using tickets.