large_client_header_buffers directive is ignored in server context

[Andrey Kartashov]

large_client_header_buffers works in http context, but ignored in server context, despite the fact it is allowed there.

To reproduce on clean installation:

Add into configuration:

  server {
    listen 80;
    server_name b;
    large_client_header_buffers 4 64k;
  }

Make sure hostname "b" resolves to your IP (via hosts file, f.e.)

Try curl with URL bigger then default buffer size (8k), f.e.:

curl -I http://b?`perl -e "print 'A' x 10000;"`

Expected reply:

HTTP/1.1 200 OK

Actual reply

HTTP/1.1 414 Request-URI Too Large

[Roman Arutyunyan]

Looks like you have more severs in configuration, and this server is not default. The default one probably does not have the large_client_header_buffers directive.

When parsing a plain HTTP request, nginx starts with the default server configuration. Only after parsing the Host header, nginx switches to a configuration of a particular virtual server. In your case the HTTP request line is obviously parsed before any header and so the default server configuration is used.

However, when using HTTPS, nginx may switch to the target virtual server during SSL handshake if client sends SNI.

[Andrey Kartashov]

Thanks for your answer, it is more clear for me now.

Well, this behavior is not reflected in documentation anyhow and this brings unnecessary confusion for people who don't understand HTTP(S) internals.

Documentation ​https://nginx.ru/en/docs/http/request_processing.html does not clearly state what happens before nginx processes Host header.

I would add some explanation for this subtle nuance like below into ​https://nginx.ru/en/docs/http/ngx_http_core_module.html for client_header_buffer_size, large_client_header_buffers and client_header_timeout (and mb some others):

---

While these directive can be specified in a server context, one should understand that it will (not) work differently for HTTP and HTTPS/SNI.

Due to the definition of HTTP protocol (see ​https://tools.ietf.org/html/rfc2616#section-5 ), nginx should read and save request URI _before_ the Host header, so it must take buffer configuration from http context or from default server context. So, in case of pure HTTP, large_client_header_buffers is effectively ignored in non-default server

For HTTPS connections with SNI, nginx is able to determine which server context to use beforehand, so buffer configuration is taken from corresponding server clause

[Maxim Dounin]

For HTTP, large_client_header_buffers can be used not only to parse URI, but also to parse headers. As already written by arut, settings specific to a non-default server are applied after parsing the Host header, so it is used to parse most of the headers. In most practical cases it is the Cookie header that overflows buffers, so large_client_header_buffers works as expected even in non-default servers.

Trying to improve the documentation might be tricky here, especially assuming a reader does not understand HTTP / HTTPS internals, as basically all settings use values from the default server configuration before nginx is able to determine which non-default server should be used. And the exact moment when switching to a different server is possible depends on various protocol and implementation details.

We've already tried to add explanations like this in various directives (​merge_slashes, ​underscores_in_headers, ​ignore_invalid_headers). I can't say it is a good practice though, as it is not really accurate, and might instead be a source of confusion.

It is also not clear when to add such a warning, and when not to. For example, consider ​connection_pool_size. Obviously enough, it will use the default server value, always. And explicitly explaining this fact might not be the best idea.

[Andrey Kartashov]

May be better option will be to add separate section for this into ​https://nginx.ru/en/docs/http/request_processing.html and then add link to this section into description of affected directives.

---

## How and when nginx determines which virtual server to use

As it is mentioned in other sections, nginx tests the request’s header field “Host” to determine which server the request should be routed to. But this is true only for HTTP connections.

For HTTPS connections with SNI, nginx is able to determine server name earlier, during TLS handshake.

Important thing here to understand is that nginx should process connection and part of the request before it is able to determine server name. Due to this fact, some directives specified in a server context may take effect only at a later stage. Before this, nginx takes configuration from default server context or from http context if default server does not have it.

Simple exampe is large_client_header_buffers directive, which controls buffer size for reading request's header. In the very beginning of the request, nginx uses value defined in default server for buffer size. Once nginx reaches the Host header, it updates buffer size with the value from arbitrary virtual server and the rest of the headers are processed with this server's configuration.

[Maxim Dounin]

See also #1987 to emphasize "might instead be a source of confusion" in comment:4 above.

[Maxim Dounin]

Documented a while ago in [c56adb7148a4/nginx_org] and [691de23367a7/nginx_org], see ​Virtual server selection section in the ​Server names article. Thanks to all involved, closing this.