Opened 19 months ago

Closed 15 months ago

Last modified 2 weeks ago

#1520 closed defect (invalid)

HTTP/2 connection dropped when URL has large numbers of same parameter

Reported by: alubbock@… Owned by:
Priority: critical Milestone:
Component: other Version: 1.13.x
Keywords: Cc:
uname -a: Linux a3759fdce72a 4.2.8-200.fc22.x86_64 #1 SMP Tue Dec 15 16:50:23 UTC 2015 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.13.11 built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) built with OpenSSL 1.1.0f 25 May 2017 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.13.11/debian/debuild-base/nginx-1.13.11=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

Description

HTTP/2 connections are being dropped where the URI request contains a large number of repeats of the same parameter. Verified on nginx 1.13.11, and can even be reproduced using a request against www.nginx.com at the time of writing.

curl 'https://www.nginx.com/?c=3860&c=1155&c=3861&c=3862&c=3863&c=3864&c=1159&c=3865&c=3866&c=1162&c=3867&c=3868&c=1166&c=1167&c=3869&c=1168&c=1169&c=1170&c=21&c=1171&c=1172&c=23&c=3870&c=1178&c=3871&c=1179&c=1180&c=3872&c=1183&c=1185&c=3873&c=3874&c=3875&c=3876&c=3877&c=3878&c=3879&c=1194&c=3880&c=3881&c=1203&c=3882&c=1206&c=3883&c=3884&c=3885&c=1210&c=3886&c=3887&c=1212&c=1213&c=3888&c=3889&c=3890&c=3891&c=3892&c=3893&c=3894&c=3895&c=3896&c=3897&c=3898&c=3899&c=3900&c=3901&c=3902&c=3903&c=8&c=3904&c=3905&c=3906&c=3907&c=3908&c=3909&c=3910&c=3911&c=3912&c=3913&c=3914&c=3915&c=3916&c=3917&c=3918&c=3919&c=3920&c=3921&c=3922&c=3923&c=3924&c=3925&c=3926&c=3927&c=3928&c=3929&c=3930&c=3931&c=1224&c=1228&c=1229&c=1231&c=3932&c=3933&c=3934&c=3935&c=3936&c=3937&c=3938&c=3939&c=3940&c=3941&c=3942&c=3943&c=3944&c=3945&c=3946&c=3947&c=3948&c=3949&c=3950&c=3951&c=3952&c=3953&c=3954&c=3955&c=3956&c=3957&c=3958&c=3959&c=3960&c=3961&c=3962&c=3963&c=3964&c=1266&c=3965&c=3966&c=3967&c=3968&c=3969&c=3970&c=3971&c=3972&c=3973&c=3974&c=3975&c=3976&c=3977&c=3978&c=3979&c=3980&c=3981&c=3982&c=3983&c=3984&c=3985&c=3986&c=3987&c=3988&c=3989&c=1320&c=3990&c=1322&c=3991&c=3992&c=3993&c=3994&c=3995&c=3996&c=3997&c=3998&c=3999&c=1335&c=36&c=4000&c=4001&c=1340&c=1341&c=1342&c=4002&c=4003&c=1348&c=1349&c=4004&c=4005&c=4006&c=4007&c=4008&c=1357&c=1358&c=1359&c=4009&c=1363&c=4010&c=4011&c=4012&c=4013&c=4014&c=1390&c=4015&c=4016&c=1394&c=4017&c=4018&c=4019&c=4020&c=4021&c=4022&c=1403&c=4023&c=1406&c=4024&c=4025&c=4026&c=4027&c=4028&c=4029&c=4030&c=4031&c=4032&c=1439&c=3&c=4033&c=4034&c=1449&c=1450&c=1451&c=4035&c=4036&c=1452&c=1453&c=4037&c=1455&c=4038&c=1456&c=1457&c=1458&c=4039&c=1460&c=4040&c=4041&c=4042&c=4043&c=1462&c=17&c=4044&c=4045&c=4046&c=18&c=4047&c=4048&c=4049&c=4050&c=4051&c=4052&c=4053&c=4054&c=4055&c=4056&c=4057&c=4058&c=4059&c=4060&c=4061&c=4062&c=1471&c=4063&c=4064&c=4065&c=4066&c=4067&c=1473&c=4068&c=4069&c=4070&c=4071&c=4072&c=4073&c=1482&c=1483&c=4074&c=4075&c=4076&c=4077&c=4078&c=4079&c=4080&c=4081&c=4082&c=4083&c=4084&c=4085&c=4086&c=4087&c=4088&c=4089&c=4090&c=4091&c=4092&c=4093&c=4094&c=4095&c=4096&c=4097&c=1489&c=4098&c=4099&c=4100&c=4101&c=4102&c=4103&c=1497&c=4104&c=4105&c=4106&c=4107&c=4108&c=4109&c=4110&c=4111&c=1501&c=1511&c=1512&c=4112&c=4113&c=4114&c=4115&c=4116&c=4117&c=4118&c=4119&c=4120&c=4121&c=1537&c=4122&c=4123&c=4124&c=4125&c=4126&c=4127&c=4128&c=4129&c=4130&c=4131&c=4132&c=4133&c=4134&c=4135&c=4136&c=4137&c=4138&c=1557&c=4139&c=4140&c=4141&c=4142&c=1563&c=4143&c=4144&c=4145&c=4146&c=4147&c=4148&c=4149&c=4150&c=1580&c=4151&c=4152&c=4153&c=4154&c=1583&c=4155&c=4156&c=4157&c=1588&c=1590&c=4158&c=4159&c=4160&c=4161&c=4162&c=4163&c=4164&c=4165&c=4166&c=4167&c=4168&c=4169&c=4170&c=4171&c=4172&c=4173&c=4174&c=4175&c=4176&c=4177&c=4178&c=4179&c=4180&c=4181&c=4182&c=4183&c=4184&c=4185&c=4186&c=1612&c=1613&c=4187&c=4188&c=4189&c=4190&c=4191&c=4192&c=4193&c=4194&c=4195&c=4196&c=4197&c=4198&c=4199&c=4200&c=4201&c=4202&c=4203&c=4204&c=4205&c=4206&c=4207&c=4208&c=4209&c=4210&c=4211&c=4212&c=4213&c=4214&c=1659&c=4215&c=4216&c=4217&c=4218&c=4219&c=4220&c=4221&c=4222&c=4223&c=4224&c=1673&c=4225&c=4226&c=4227&c=1680&c=1681&c=1684&c=4228&c=5&c=4229&c=4230&c=4231&c=6&c=7&c=1695&c=4232&c=4233&c=4234&c=4235&c=4236&c=30&c=4237&c=4238&c=4239&c=4240&c=4241&c=4242&c=4243&c=4244&c=4245&c=4246&c=1716&c=1718&c=1719&c=4247&c=4248&c=1723&c=4249&c=4250&c=4251&c=4252&c=4253&c=4254&c=4255&c=4256&c=4257&c=4258&c=4259&c=4260&c=4261&c=4262&c=4263&c=4264&c=1747&c=1749&c=1755&c=4265&c=4266&c=4267&c=4268&c=4269&c=4270&c=4271&c=4272&c=4273&c=4274&c=4275&c=4276&c=4277&c=4278&c=4279&c=4280&c=4281&c=4282&c=4283&c=4284&c=4285&c=4286&c=4287&c=4288&c=4289&c=4290&c=4291&c=4292&c=4293&c=4294&c=4295&c=4296&c=4297&c=4298&c=4299&c=4300&c=4301&c=4302&c=4303&c=4304&c=4305&c=4306&c=4307&c=4308&c=4309&c=4310&c=4311&c=4312&c=4313&c=4314&c=4315&c=4316&c=4317&c=4318&c=4319&c=4320&c=4321&c=4322&c=4323&c=4324&c=4325&c=4326&c=4327&c=4328&c=4329&c=4330&c=4331&c=4332&c=4333&c=4334&c=4335&c=4336&c=4337&c=4338&c=4339&c=4340&c=4341&c=4342&c=4343&c=4344&c=4345&c=4346&c=4347&c=4348&c=4349&c=4350&c=4351&c=4352&c=4353&c=4354&c=4355&c=4356&c=4357&c=4358&c=4359&c=4360&c=4361&c=4362&c=4363&c=4364&c=4365&c=4366&c=4367&c=4368&c=4369&c=4370&c=4371&c=4372&c=1916&c=1917&c=1918&c=4373&c=4374&c=4375&c=4376&c=4377&c=4378&c=4379&c=4380&c=4381&c=4382&c=4383&c=1938&c=4384&c=4385&c=1942&c=4386&c=4387&c=4388&c=4389&c=4390&c=4391&c=4392&c=4393&c=4394&c=4395&c=4396&c=4397&c=4398&c=4399&c=4400&c=4401&c=4402&c=4403&c=4404&c=4405&c=4406&c=4407&c=4408&c=4409&c=4410&c=4411&c=4412&c=1973&c=4413&c=4414&c=1984&c=4415&c=4416&c=1987&c=4417&c=4418&c=1989&c=4419&c=4420&c=4421&c=4422&c=4423&c=4424&c=4425&c=4426&c=1998&c=1999&c=2000&c=4427&c=4428&c=4429&c=4430&c=4431&c=4432&c=4433&c=4434&c=2013&c=4435&c=2016&c=4436&c=4437&c=4438&c=4439&c=4440&c=4441&c=2032&c=4442&c=2033&c=2034&c=2035&c=2036&c=4443&c=4444&c=4445&c=4446&c=2040&c=4447&c=4448&c=4449&c=4450&c=4451&c=4452&c=4453&c=4454&c=4455&c=32&c=4456&c=33&c=4457&c=4458&c=4459&c=24&c=4460&c=4461&c=4462&c=4463&c=4464&c=4465&c=4466&c=4467&c=4468&c=4469&c=2076&c=4470&c=4471&c=4472&c=4473&c=4474&c=4475&c=4476&c=4477&c=4478&c=4479&c=4480&c=4481&c=4482&c=4483&c=4484&c=4485&c=4486&c=4487&c=4488&c=4489&c=4490&c=4491&c=4492&c=4493&c=4494&c=4495&c=4496&c=4497&c=4498&c=4499&c=4500&c=4501&c=4502&c=4503&c=4504&c=4505&c=4506&c=4507&c=4508&c=4509&c=4510&c=4511&c=4512&c=4513&c=4514&c=4515&c=4516&c=4517&c=4518&c=4519&c=4520&c=4521&c=4522&c=4523&c=4524&c=4525&c=2093&c=2102&c=4526&c=4527&c=4528&c=4529&c=4530&c=4531&c=4532&c=4533&c=4534&c=4535&c=4536&c=2109&c=2110&c=2111&c=4537&c=2113&c=2114&c=2116&c=2117&c=2118&c=2119&c=2121&c=4538&c=2122&c=2130&c=2133&c=4539&c=2137&c=4540&c=4541&c=2141&c=2142&c=2143&c=4542&c=4543&c=4544&c=2148&c=4545&c=4546&c=4547&c=4548&c=4549&c=4550&c=4551&c=4552&c=4553&c=4554&c=4555&c=4556&c=4557&c=4558&c=4559&c=4560&c=4561&c=4562&c=4563&c=4564&c=4565&c=4566&c=4567&c=4568&c=4569&c=2178&c=2184&c=4570&c=4571&c=4572&c=4573&c=4574&c=4575&c=4576&c=4577&c=4578&c=4579&c=4580&c=4581&c=4582&c=2198&c=4583&c=4584&c=4585&c=4586&c=19&c=26&c=22&c=29&c=25&c=4587&c=4588&c=4589&c=2215&c=4590&c=4591&c=4592&c=4593&c=4594&c=4595&c=4596&c=4597'

The above returns curl: (56) Unexpected EOF. The connection is dropped in Chrome and Firefox, too. This only seems to occur with HTTP/2 - the connection succeeds with HTTP/1.1. I've tried raising all the various buffer parameters with no success. This issue can be reproduced using the current nginx:mainline Docker image.

Change History (9)

comment:1 Changed 19 months ago by vbart

Have you checked the error log? In most cases it contains the reason.

Also, it's not clear what "buffer parameters" you've tried to rise, but your request is bigger than the default http2_max_field_size value.

See the documentation:
http://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_field_size

Last edited 19 months ago by vbart (previous) (diff)

comment:2 Changed 19 months ago by alubbock@…

Raising both http2_max_field_size and http2_max_header_size fixes the issue. I hadn't realized HTTP/2 headers were handled separately in the configuration. Thanks for your help.

The only remaining issue is: shouldn't the server return HTTP 414, rather than dropping the connection? Any other concurrent requests over the same HTTP/2 connection are also dropped, which may be confusing for multi-threaded client applications.

comment:3 Changed 19 months ago by vbart

Since headers in HTTP/2 protocol are encoded using stateful compression algorithm, it's impossible to continue maintaining connection if there's any problem with handling headers in a request (e.g. limits are reached).

Note that nginx doesn't just drop the connection, but it sends a GOAWAY frame with ENHANCE_YOUR_CALM protocol error.

comment:4 Changed 15 months ago by mdounin

  • Resolution set to invalid
  • Status changed from new to closed

comment:5 Changed 7 months ago by mdounin

See also #1749.
Other related tickets: #1420, #1387, #1027, #793.

comment:6 Changed 4 months ago by mdounin

See also #1800.

comment:7 Changed 4 months ago by mdounin

See also #1508.

comment:8 Changed 4 months ago by mdounin

See also #1802.

comment:9 Changed 2 weeks ago by mdounin

See also #1866.

Last edited 2 weeks ago by mdounin (previous) (diff)
Note: See TracTickets for help on using tickets.