Opened 7 years ago
Closed 23 months ago
#1523 closed defect (fixed)
large_client_header_buffers directive is ignored in server context
Reported by: | Andrey Kartashov | Owned by: | |
---|---|---|---|
Priority: | minor | Milestone: | |
Component: | documentation | Version: | 1.13.x |
Keywords: | Cc: | ||
uname -a: | Linux sshct 4.14.32-coreos #1 SMP Tue Apr 3 05:21:26 UTC 2018 x86_64 GNU/Linux | ||
nginx -V: |
nginx version: nginx/1.10.3
built with OpenSSL 1.1.0f 25 May 2017 TLS SNI support enabled configure arguments: --with-cc-opt='-g -O2 -fdebug-prefix-map=/build/nginx-2tpxfc/nginx-1.10.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_xslt_module=dynamic --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module --add-dynamic-module=/build/nginx-2tpxfc/nginx-1.10.3/debian/modules/nginx-auth-pam --add-dynamic-module=/build/nginx-2tpxfc/nginx-1.10.3/debian/modules/nginx-dav-ext-module --add-dynamic-module=/build/nginx-2tpxfc/nginx-1.10.3/debian/modules/nginx-echo --add-dynamic-module=/build/nginx-2tpxfc/nginx-1.10.3/debian/modules/nginx-upstream-fair --add-dynamic-module=/build/nginx-2tpxfc/nginx-1.10.3/debian/modules/ngx_http_substitutions_filter_module |
Description
large_client_header_buffers
works in http context, but ignored in server context, despite the fact it is allowed there.
To reproduce on clean installation:
Add into configuration:
server { listen 80; server_name b; large_client_header_buffers 4 64k; }
Make sure hostname "b" resolves to your IP (via hosts file, f.e.)
Try curl with URL bigger then default buffer size (8k), f.e.:
curl -I http://b?`perl -e "print 'A' x 10000;"`
Expected reply:
HTTP/1.1 200 OK
Actual reply
HTTP/1.1 414 Request-URI Too Large
Change History (7)
comment:1 by , 7 years ago
comment:2 by , 7 years ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
comment:3 by , 7 years ago
Resolution: | invalid |
---|---|
Status: | closed → reopened |
Thanks for your answer, it is more clear for me now.
Well, this behavior is not reflected in documentation anyhow and this brings unnecessary confusion for people who don't understand HTTP(S) internals.
Documentation https://nginx.ru/en/docs/http/request_processing.html does not clearly state what happens before nginx processes Host header.
I would add some explanation for this subtle nuance like below into https://nginx.ru/en/docs/http/ngx_http_core_module.html for client_header_buffer_size
, large_client_header_buffers
and client_header_timeout
(and mb some others):
While these directive can be specified in a server context, one should understand that it will (not) work differently for HTTP and HTTPS/SNI.
Due to the definition of HTTP protocol (see https://tools.ietf.org/html/rfc2616#section-5 ), nginx should read and save request URI _before_ the Host header, so it must take buffer configuration from http context or from default server context. So, in case of pure HTTP, large_client_header_buffers
is effectively ignored in non-default server
For HTTPS connections with SNI, nginx is able to determine which server context to use beforehand, so buffer configuration is taken from corresponding server clause
comment:4 by , 7 years ago
Component: | other → documentation |
---|
For HTTP, large_client_header_buffers
can be used not only to parse URI, but also to parse headers. As already written by arut, settings specific to a non-default server are applied after parsing the Host
header, so it is used to parse most of the headers. In most practical cases it is the Cookie
header that overflows buffers, so large_client_header_buffers
works as expected even in non-default servers.
Trying to improve the documentation might be tricky here, especially assuming a reader does not understand HTTP / HTTPS internals, as basically all settings use values from the default server configuration before nginx is able to determine which non-default server should be used. And the exact moment when switching to a different server is possible depends on various protocol and implementation details.
We've already tried to add explanations like this in various directives (merge_slashes, underscores_in_headers, ignore_invalid_headers). I can't say it is a good practice though, as it is not really accurate, and might instead be a source of confusion.
It is also not clear when to add such a warning, and when not to. For example, consider connection_pool_size. Obviously enough, it will use the default server value, always. And explicitly explaining this fact might not be the best idea.
comment:5 by , 7 years ago
May be better option will be to add separate section for this into https://nginx.ru/en/docs/http/request_processing.html and then add link to this section into description of affected directives.
## How and when nginx determines which virtual server to use
As it is mentioned in other sections, nginx tests the request’s header field “Host” to determine which server the request should be routed to. But this is true only for HTTP connections.
For HTTPS connections with SNI, nginx is able to determine server name earlier, during TLS handshake.
Important thing here to understand is that nginx should process connection and part of the request before it is able to determine server name. Due to this fact, some directives specified in a server context may take effect only at a later stage. Before this, nginx takes configuration from default server context or from http context if default server does not have it.
Simple exampe is large_client_header_buffers directive, which controls buffer size for reading request's header. In the very beginning of the request, nginx uses value defined in default server for buffer size. Once nginx reaches the Host header, it updates buffer size with the value from arbitrary virtual server and the rest of the headers are processed with this server's configuration.
comment:6 by , 4 years ago
comment:7 by , 23 months ago
Resolution: | → fixed |
---|---|
Status: | reopened → closed |
Documented a while ago in [c56adb7148a4/nginx_org] and [691de23367a7/nginx_org], see Virtual server selection section in the Server names article. Thanks to all involved, closing this.
Looks like you have more severs in configuration, and this server is not default. The default one probably does not have the large_client_header_buffers directive.
When parsing a plain HTTP request, nginx starts with the default server configuration. Only after parsing the Host header, nginx switches to a configuration of a particular virtual server. In your case the HTTP request line is obviously parsed before any header and so the default server configuration is used.
However, when using HTTPS, nginx may switch to the target virtual server during SSL handshake if client sends SNI.