Opened 2 years ago

Last modified 10 months ago

#2268 accepted defect

http2 client set both host and :authority header, server throws 400 bad request error

Reported by: xbkaishui@… Owned by:
Priority: minor Milestone:
Component: nginx-module Version: 1.19.x
Keywords: duplicate host header Cc:
uname -a: Darwin xbkaishui-2.local 19.6.0 Darwin Kernel Version 19.6.0: Mon Apr 12 20:57:45 PDT 2021; root:xnu-6153.141.28.1~1/RELEASE_X86_64 x86_64
nginx -V: nginx version: nginx/1.19.7
built by clang 12.0.0 (clang-1200.0.32.29)
built with OpenSSL 1.1.1i 8 Dec 2020 (running with OpenSSL 1.1.1k 25 Mar 2021)
TLS SNI support enabled
configure arguments: --prefix=/usr/local/Cellar/nginx/1.19.7 --sbin-path=/usr/local/Cellar/nginx/1.19.7/bin/nginx --with-cc-opt='-I/usr/local/opt/pcre/include -I/usr/local/opt/openssl@1.1/include' --with-ld-opt='-L/usr/local/opt/pcre/lib -L/usr/local/opt/openssl@1.1/lib' --conf-path=/usr/local/etc/nginx/nginx.conf --pid-path=/usr/local/var/run/nginx.pid --lock-path=/usr/local/var/run/nginx.lock --http-client-body-temp-path=/usr/local/var/run/nginx/client_body_temp --http-proxy-temp-path=/usr/local/var/run/nginx/proxy_temp --http-fastcgi-temp-path=/usr/local/var/run/nginx/fastcgi_temp --http-uwsgi-temp-path=/usr/local/var/run/nginx/uwsgi_temp --http-scgi-temp-path=/usr/local/var/run/nginx/scgi_temp --http-log-path=/usr/local/var/log/nginx/access.log --error-log-path=/usr/local/var/log/nginx/error.log --with-compat --with-debug --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-ipv6 --with-mail --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module

Description

when use http2 client. we both set host and :authority header. nginx server throw 400 bad request. the error log is
*1 client sent duplicate host header: "host: xxx", previous value: "host: 127.0.0.1:27710" while reading client request headers, client: 127.0.0.1, server: _, host: "127.0.0.1:27710"

this is very confused. need some help.

Change History (10)

comment:1 by Sergey Kandaurov, 2 years ago

Status: newaccepted

While HTTP/2 forbids requests with different Host header and :authority pseudo-header, requests with identical values are permitted. Looks like an oversight from 4f18393a1d51 in 1.17.9.

comment:2 by Maxim Dounin, 2 years ago

The HTTP/2 code pretends that ":authority" pseudo-header is the "Host" header, see here, misusing the "Host" header processing. This probably should be rewritten to treat it similarly to the authority from the request line instead (and the "Host" header probably needs to be checked against authority, if present), though I would expected this to break a lot of misconfigured setups trying to use $http_host instead of $host.

Either way, it looks like in the particular case the error is correct, since previous value, "127.0.0.1:27710", does not match the new one, "xxx". That's something to be fixed in the client.

What's the client you are using?

in reply to:  2 comment:3 by xbkaishui@…, 2 years ago

I am using grpc-java client, sorry i don't know the rule that ":authority" and Host is the same. but it is ok in less than nginx1.17, the host header in our case is for routing request with http1 and http2, I know your point, thanks

Replying to Maxim Dounin:

The HTTP/2 code pretends that ":authority" pseudo-header is the "Host" header, see here, misusing the "Host" header processing. This probably should be rewritten to treat it similarly to the authority from the request line instead (and the "Host" header probably needs to be checked against authority, if present), though I would expected this to break a lot of misconfigured setups trying to use $http_host instead of $host.

Either way, it looks like in the particular case the error is correct, since previous value, "127.0.0.1:27710", does not match the new one, "xxx". That's something to be fixed in the client.

What's the client you are using?

in reply to:  1 comment:4 by xbkaishui@…, 2 years ago

Got it, so is it make sense for this change? I think it is a break change for http1 and http2

Replying to Sergey Kandaurov:

While HTTP/2 forbids requests with different Host header and :authority pseudo-header, requests with identical values are permitted. Looks like an oversight from 4f18393a1d51 in 1.17.9.

comment:5 by Maxim Dounin, 2 years ago

I am using grpc-java client

I'm pretty sure grpc-java, even if for some reason it will try to send both ":authority" and "Host", will never use different values for these headers. Could you please confirm that the "Host" header is something manually added by your code which uses grpc-java?

in reply to:  5 comment:6 by xbkaishui@…, 2 years ago

Yes, I am use custom intercepter add host header
code gist like this https://gist.github.com/xbkaishui/49a052a017ee8441e7a6833a65874123
i am not very familiar with grpc-java. is there any other way to set host header?
Thanks

Replying to Maxim Dounin:

I am using grpc-java client

I'm pretty sure grpc-java, even if for some reason it will try to send both ":authority" and "Host", will never use different values for these headers. Could you please confirm that the "Host" header is something manually added by your code which uses grpc-java?

Last edited 2 years ago by xbkaishui@… (previous) (diff)

comment:7 by Maxim Dounin, 2 years ago

Yes, I am use custom intercepter add host header

Thanks for confirming. Trivial fix would be to remove it, so only the ":authority" header as sent by grpc-java will be present in requests.

is there any other way to set host header?

There is no need to. The ":authority" pseudo-header is equivalent and should be used instead of the "Host" header in HTTP/2, see RFC 7540.

in reply to:  7 comment:8 by xbkaishui@…, 2 years ago

Ok, got it. thanks

Replying to Maxim Dounin:

Yes, I am use custom intercepter add host header

Thanks for confirming. Trivial fix would be to remove it, so only the ":authority" header as sent by grpc-java will be present in requests.

is there any other way to set host header?

There is no need to. The ":authority" pseudo-header is equivalent and should be used instead of the "Host" header in HTTP/2, see RFC 7540.

in reply to:  7 ; comment:9 by bullerdu@…, 10 months ago

Replying to Maxim Dounin:

There is no need to. The ":authority" pseudo-header is equivalent and should be used instead of the "Host" header in HTTP/2, see RFC 7540.

Clients MUST NOT generate a request with a Host header field that differs from the ":authority" pseudo-header field. A server SHOULD treat a request as malformed if it contains a Host header field that identifies an entity that differs from the entity in the ":authority" pseudo-header field. see RFC9113:https://www.rfc-editor.org/rfc/rfc9113#section-8.3.1

so, http2 client can set "Host" header and ":authority" pseudo-header with the same value?

in reply to:  9 comment:10 by Maxim Dounin, 10 months ago

Replying to bullerdu@…:

so, http2 client can set "Host" header and ":authority" pseudo-header with the same value?

This is not forbidden by RFC, see comment:1. This won't work with nginx though (and that's why this ticket is not closed), and not really needed (since the ":authority" pseudo-header is enough).

Note: See TracTickets for help on using tickets.