Chunked transfer encoding with Live video stream does not clean tmp files

[unifiedstreaming@…]

Hi Maxim,

When using 'chunked transfer encoding' Nginx writes the POSTed content to a tmp file (or files).

This will fill up the disk with a large file, but also when a 24/7 Live video stream is sent using chunked transfer encoding. It basically is a a long standing POST where chunks of data are sent within that POST, so the connection is not closed until the stream stops and as it is a 24/7 live linear stream this equals to 'never'.

Nginx therefore cannot be used with a Live when the ingest is handled with a 'request body filter'.

I attach the reference module that shows this behaviour as well as a curl line to do the post.

Kind regards,

Dirk

Log:

2017/07/05 12:57:33 [warn] 25407#0: *4 a client request body is buffered to a temporary file /usr/local/nginx/client_body_temp/0000000002, client: 127.0.0.1, server: localhost, request: "POST /large-file-pretending-to-be-video.ismv HTTP/1.1", host: "127.0.0.1:8080"

2017/07/05 12:57:33 [crit] 25407#0: *4 pwrite() "/usr/local/nginx/client_body_temp/0000000002" failed (28: No space left on device), client: 127.0.0.1, server: localhost, request: "POST /large-file-pretending-to-be-video.ismv HTTP/1.1", host: "127.0.0.1:8080"

[unifiedstreaming@…]

A zip file containing module (src, makefile and config) as well as nginx.conf and curl script

[unifiedstreaming@…]

Please note that the following line:

make sure this flag is up, otherwise we may be missing the last part of the body
r->request_body_in_file_only = 1;

does not matter.

When left out of ngx_http_catch_body_filter_module.c the behaviour is still the same as outlined above.

Is there any combination of settings from ngx_http_request.h as in:

unsigned request_body_in_single_buf:1;
unsigned request_body_in_file_only:1;
unsigned request_body_in_persistent_file:1;
unsigned request_body_in_clean_file:1;
unsigned request_body_file_group_access:1;
unsigned request_body_file_log_level:3;
unsigned request_body_no_buffering:1;

or from the 'client_body_...' configuration as in:

​http://nginx.org/en/docs/http/ngx_http_core_module.html

that will have effect on the tmp file writing so that

a) the tmp file does not grow infinitely appending posted data
b) remove created tmp files while the POST is running

As a question, why is a tmp file use any way?

The request body filter should handle incoming data and it should decide whether or not to use a tmp file

Please advise.

[Maxim Dounin]

By default, the request body is fully read by nginx if it is needed for further processing. As long as a request body fits into client_body_buffer_size, it is stored in memory. If the buffer size is not enough (or writing to a file is explicitly requested for some reason, as in your example), the body is written to a temporary file under the client_body_temp_path directory. The maximum size of the body, and accordingly the maximum size of the temporary file, can be controlled using the client_max_body_size directive.

The approach of reading the whole body before doing anything allows nginx to keep working with requests simple regardless of whether they have bodies or not, including ability to freely retry requests to different upstream servers and so on. It also makes sure that backend processes are not occupied when the body is not yet arrived from the client, and we are just waiting for the body - much like when nginx buffers a response from a backend server to free the process as fast as possible.

If you don't want the body to be fully read before processing, it is possible to do so when supported by appropriate modules and underlying protocols. Notably, the proxy module has the ​proxy_request_buffering directive to pass the request body to an upstream once it arrives, without trying to fully read it. Note though that this is a special mode and it implies various limitations, including:

• Needs either body size known in advance, or a protocol which supports unspecified body size. Notably, this means that chunked requests will only work with proxy_http_version 1.1.
• Various limitations on error handling and redirections: the request cannot be passed to a different server.

If you have further question, please don't hesitate to use ​existing support options.