Opened 6 years ago

Closed 4 years ago

Last modified 3 years ago

#251 closed enhancement (fixed)

Option to disable upload buffers

Reported by: timwhite88@… Owned by: somebody
Priority: minor Milestone:
Component: nginx-core Version: 1.2.x
Keywords: buffer uploaded Cc:
uname -a: Linux vmsrv-lino 3.4.2-x86_64-linode25 #2 SMP Tue Jun 26 16:10:49 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.2.4 TLS SNI support enabled configure arguments: --prefix=/etc/nginx/ --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/ --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6


Partially related to #159, the ability to disable upload buffers both for FastCGI and HTTP proxying would be very beneficial. Using the Upload module isn't always pratical, requiring a rewrite of some application code. And as a static file accelerator in front of another web server, being able to upload large files and not have it buffer would be very useful. In particular, lots of file uploaders break as the make a second request to get progress, only to find out that the file hasn't even "started uploading" as it's still buffering to Nginx.

At the end of the day, if you have to rewrite application code on the web server behind Nginx, then Nginx isn't as useful as a static file server.

Change History (13)

comment:1 Changed 6 years ago by timmyschweer@…

We're using nginx to provide an typo3 page and a lot of javascript / PHP stuff. We started a new service using java servlets contained in tomcat / jetty. The nginx upload buffer messes up our upload servlet on large files so please provide the functionality described in this ticket.

comment:2 Changed 6 years ago by

Guys, this is a huge issue for us. Every uploaded file needs to be fully buffered to memory or disk prior to sending it to the backend server.
This is both a load & latency issue.
This makes using nginx as a reverse proxy for multiple users simply impractical.
I think this issue's priority is far higher than minor.

Do you have any idea on when this issue is solved?

comment:3 Changed 5 years ago by

Am I right to believe the upload buffering was put in place to prevent the blocking of workers in traditional backends? Newer nonblocking backends (like, say, node.js) do not require these measures as the requests are always streamed in. It would be great to have the option to disable this feature when using a newer backend that does not require nginx to babysit its ability to stream uploads.

comment:4 Changed 5 years ago by

We've run into similar issues with this when using nginx as a server for Ceph. The machine running nginx does not necessarily have sufficient disk space to buffer the entire upload, nor is that something we want it to do.

I'm hoping now that we have control of this for downloads (via fastcgi_buffering), that this is something that will be implemented for uploads as well.

I did find a patch that does this on the mailing list, so it seems feasible: . Also, tengine has support for this (via fastcgi_request_buffering)

comment:5 Changed 4 years ago by derhein@…

As explained in the mailing list post, the functionality has been added to
the tengine-fork of nginx.

Is there any reason not to integrate this feature back into nginx, or
did just nobody put in a pull request, yet?

comment:6 Changed 4 years ago by maxim

  • Status changed from new to accepted

We do have this feature in our roadmap for 1.7.

comment:7 Changed 4 years ago by mcarbonneaux@…

in what 1.7.x while be released ?

comment:8 Changed 4 years ago by mikekennewick@…

Last edited 4 years ago by mikekennewick@… (previous) (diff)

comment:9 Changed 4 years ago by

Any news on this enhancement?
Can someone of the NGINX team say when this enhancement will be deployed?

comment:10 Changed 4 years ago by maxim

ETA is this quarter.

comment:11 Changed 4 years ago by lennartbastian@…

was this removed from the 1.7 roadmap? is it still planned to be released in the next several weeks?

comment:12 Changed 4 years ago by maxim

It wasn't:

The feature was implemented in 1.7.11 with some fixes in 1.7.12.

comment:13 Changed 4 years ago by mdounin

  • Resolution set to fixed
  • Status changed from accepted to closed
Note: See TracTickets for help on using tickets.