Opened 8 years ago
Last modified 8 years ago
#969 new enhancement
proxy module does not honour proxy_max_temp_file_size on cacheable responses
| Reported by: | Owned by: | ||
|---|---|---|---|
| Priority: | minor | Milestone: | |
| Component: | nginx-module | Version: | 1.9.x |
| Keywords: | slice proxy cache temp | Cc: | |
| uname -a: | Linux 1dcda8eab5c1 4.4.8-boot2docker #1 SMP Mon Apr 25 21:57:27 UTC 2016 x86_64 Linux | ||
| nginx -V: |
nginx version: nginx/1.9.15
built by gcc 5.3.0 (Alpine 5.3.0) built with OpenSSL 1.0.2g 1 Mar 2016 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_auth_request_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_perl_module=dynamic --with-threads --with-stream --with-stream_ssl_module --with-http_slice_module --with-mail --with-mail_ssl_module --with-file-aio --with-http_v2_module --with-ipv6 |
||
Description
In the ngx_http_proxy module, there is a directive: proxy_max_temp_file_size - it is intended to limit the size of buffered files on disk. There is a caveat:
"This restriction does not apply to responses that will be cached or stored on disk."
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_max_temp_file_size
Why is this the case? I have an issue when using the slice module - I'm proxying a large request, and Nginx sends smaller range requests to the proxied server, for ~16MB hunks of the resource. Sometimes the proxied server misbehaves, and responds with a ranged response from the offset, till the end of the very large file.
Nginx will continue to 'temporarily' buffer this file to disk, resulting in a blocked downstream request, and an ever increasing use of the temporary proxy storage. The upstream file is multiple terabytes - Nginx should honor this directive and terminate the upstream range response when it receives over the max size for buffered files.
The
proxy_max_temp_file_sizeis intended to limit maximum size of disk-based buffer. But when nginx is going to cache the response, it needs full response to be written to disk - not to buffer something, but to store the response. And hence this restriction is ignored.There are no plans to change the current behaviour, though we can consider adding an option to limit maximum size of responses to be cached. Right now you can use proxy_no_cache with appropriate checks of
$upstream_http_content_lengthto avoid caching large responses (assuming Content-Length of a response is know in advance).