Opened 8 months ago

#2548 new defect

Worker infinite loop in ngx_http_do_read_client_request_body()

Reported by: anight@… Owned by:
Priority: minor Milestone:
Component: documentation Version: 1.25.x
Keywords: Cc: anight@…
uname -a: Linux server 6.2.0-32-generic #32-Ubuntu SMP PREEMPT_DYNAMIC Mon Aug 14 10:03:50 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.25.2
built by gcc 12.3.0 (Ubuntu 12.3.0-1ubuntu1~23.04)
built with OpenSSL 3.0.8 7 Feb 2023
TLS SNI support enabled
configure arguments: --with-debug --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/ --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g3 -O0'


Consider the following location configuration:

        location = @grpcweb {
                rewrite ^ $orig_grpc_uri break;
                proxy_redirect off;
                proxy_buffering off;
                proxy_request_buffering off;
                allow all;
                proxy_http_version 1.1;
                proxy_set_header Host $http_host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Host $server_name;
                proxy_set_header X-Forwarded-Proto $scheme;
                client_body_buffer_size 0;
                client_max_body_size 0;
                proxy_max_temp_file_size 0;
                proxy_read_timeout 18000;
                proxy_send_timeout 18000;
                gzip off;

I figured out that in my case ngx_http_do_read_client_request_body() is called with zero sized buffer (rb->buf->start == rb->buf->end) and as a result it loops endlessly logging "http client request body rest 3" message thousands times a second.

The zero-sized buffer is created earlier in ngx_http_read_client_request_body():

    /* TODO: honor r->request_body_in_single_buf */

    if (!r->headers_in.chunked && rb->rest < size) {
        size = (ssize_t) rb->rest;

        if (r->request_body_in_single_buf) {
            size += preread;

        if (size == 0) {

    } else {
        size = clcf->client_body_buffer_size; // zero-sized buffer created here

    rb->buf = ngx_create_temp_buf(r->pool, size);
    if (rb->buf == NULL) {
        goto done;
    r->read_event_handler = ngx_http_read_client_request_body_handler;
    r->write_event_handler = ngx_http_request_empty_handler;

    rc = ngx_http_do_read_client_request_body(r);

So as I understand in my case the whole request didn't fit r->headers_in by 3 bytes and as result rb->rest was 3.

When I removed "client_body_buffer_size 0;" from configuration the bug disappeared.

I blindly copied the set of proxied location directives from some page on stackoverflow and didn't realize "client_body_buffer_size 0;" would cause endless loop in a worker process.

Change History (0)

Note: See TracTickets for help on using tickets.