Opened 7 years ago
Last modified 7 years ago
#1408 new defect
Big file download via php-fpm on fast networks eats up memory
Reported by: | Owned by: | ||
---|---|---|---|
Priority: | minor | Milestone: | |
Component: | nginx-core | Version: | 1.13.x |
Keywords: | Cc: | ||
uname -a: | Linux xxxx 4.4.0-51-generic #72-Ubuntu SMP Thu Nov 24 18:29:54 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | ||
nginx -V: |
nginx version: nginx/1.13.6
built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5) built with OpenSSL 1.0.2g 1 Mar 2016 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie' --add-module=/opt/nginx-1.13.6/debian/modules/ngx_cache_purge |
Description
The issue and the (temporary) solution is described at:
https://github.com/owncloud/core/issues/29328
https://github.com/owncloud/core/pull/29400
Ubuntu 16.04 with PHP7.0
Change History (3)
comment:1 by , 7 years ago
follow-up: 3 comment:2 by , 7 years ago
Yes buffers were set to fastcgi_buffers 64 4K;
Testing with fastcgi_buffers 63 4K;
as you said 63 should be max, downloading went fine and no memory was eaten up.
Two points:
a.) The check in the changeset referenced is >64 and not >63 ...
b.) Add a documentation note about this limit in http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffers because many people do not know about this "silent" issue ...
comment:3 by , 7 years ago
Replying to mmattel@…:
a.) The check in the changeset referenced is >64 and not >63 ...
As explained above, this is because 64 fastcgi_buffers
ends up in 65 buffers in total (the fastcgi_buffer
is also used to read the body).
b.) Add a documentation note about this limit in http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffers because many people do not know about this "silent" issue ...
Adding a documentation note may make sense, yes. On the other hand, it is an implementation detail which vary between versions (and hopefully will be fully resolved in future versions).
Are you using
fastcgi_buffers 64 4K;
as in the configuration provided in the issue? If yes, please test with a lower number of buffers (at most 63, default 8 is fine).Using 64
fastcgi_buffers
as in the configuration provided will end up in total 65 buffers being used to read the response, and this makes it possible thatngx_readv_file()
will have to allocate memory which will be only freed when closing the connection, see d1bde5c3c5d2.