fastcgi: CONTENT_LENGTH parameter missing for chunked requests

[SiebelsTim@…]

When disabling fastcgi_request_buffering and receiving a request with chunked transfer encoding, nginx does not pass a CONTENT_LENGTH parameter to the fastcgi responder.

As a result, we must disable fastcgi_request_buffering entirely, rather than having nginx handle it itself regarding the request type.

The specification says:

Next the Responder application receives CGI/1.1 stdin data from the Web server over FCGI_STDIN. The application receives at most CONTENT_LENGTH bytes from this stream before receiving the end-of-stream indication. (The application receives less than CONTENT_LENGTH bytes only if the HTTP client fails to provide them, e.g. because the client crashed.)

Furthermore, the CGI specification says:

The server MUST set this meta-variable if and only if the request is accompanied by a message-body entity. The CONTENT_LENGTH value must reflect the length of the message-body after the server has removed any transfer-codings or content-codings.

Therefore, I think nginx is supposed to buffer all chunked requests and calculate the content length to pass it to the fastcgi responder.

Apache had similar problems. They solved it by buffering all chunked requests: ​https://bz.apache.org/bugzilla/show_bug.cgi?id=53332, ​https://bz.apache.org/bugzilla/show_bug.cgi?id=57087

I reproduced this with different systems and nginx versions:

Linux $hostname 5.13.0-21-generic #21-Ubuntu SMP Tue Oct 19 08:59:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

nginx version: nginx/1.21.3
built by gcc 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
built with OpenSSL 1.1.1f 31 Mar 2020
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -ffile-prefix-map=/build/nginx-q9LD4J/nginx-1.18.0=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -fPIC' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-compat --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_flv_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_secure_link_module --with-http_sub_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-stream=dynamic --with-stream_geoip_module=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-debug

Linux $hostname 5.10.0-8-amd64 #1 SMP Debian 5.10.46-4 (2021-08-03) x86_64 GNU/Linux

nginx version: nginx/1.18.0
built with OpenSSL 1.1.1k 25 Mar 2021
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -ffile-prefix-map=/build/nginx-q9LD4J/nginx-1.18.0=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -fPIC' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-compat --with-debug --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_flv_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_secure_link_module --with-http_sub_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-stream=dynamic --with-stream_geoip_module=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-headers-more-filter --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-auth-pam --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-cache-purge --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-dav-ext --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-ndk --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-echo --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-fancyindex --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-geoip2 --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/nchan --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-lua --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/rtmp --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-uploadprogress --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-upstream-fair --add-dynamic-module=/build/nginx-q9LD4J/nginx-1.18.0/debian/modules/http-subs-filter

[SiebelsTim@…]

fastcgi specification: ​http://www.mit.edu/~yandros/doc/specs/fcgi-spec.html#S6.2

CGI specification: ​https://datatracker.ietf.org/doc/html/rfc3875#section-4.1.2

[SiebelsTim@…]

References: ​https://forum.nginx.org/read.php?2,292380,292380#msg-292380
References: ​https://github.com/php/php-src/pull/7509

[Maxim Dounin]

Therefore, I think nginx is supposed to buffer all chunked requests and calculate the content length to pass it to the fastcgi responder.

That's exactly what nginx does by default, with fastcgi_request_buffering on; (which is the default). If you configure nginx to avoid request buffering, nginx expects that the FastCGI application can handle non-buffered requests, including ones without CONTENT_LENGTH present. It is certainly possible given that the FastCGI protocol provides appropriate framing for STDIN stream.

If your FastCGI application cannot handle requests without CONTENT_LENGTH or you want CONTENT_LENGTH to be present for some other reason, such as strictly following specifications, consider not changing fastcgi_request_buffering from the default.

[SiebelsTim@…]

I do not have the option to disable request buffering selectively. i.e. I have to enable buffering for *all* requests to use a spec compliant FastCGI responder. That's quite a strong limitation.