Opened 6 years ago

Closed 6 years ago

Last modified 4 years ago

#1520 closed defect (invalid)

HTTP/2 connection dropped when URL has large numbers of same parameter

Reported by: alubbock@… Owned by:
Priority: critical Milestone:
Component: other Version: 1.13.x
Keywords: Cc:
uname -a: Linux a3759fdce72a 4.2.8-200.fc22.x86_64 #1 SMP Tue Dec 15 16:50:23 UTC 2015 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.13.11
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
built with OpenSSL 1.1.0f 25 May 2017
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.13.11/debian/debuild-base/nginx-1.13.11=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

Description

HTTP/2 connections are being dropped where the URI request contains a large number of repeats of the same parameter. Verified on nginx 1.13.11, and can even be reproduced using a request against www.nginx.com at the time of writing.

curl 'https://www.nginx.com/?c=3860&c=1155&c=3861&c=3862&c=3863&c=3864&c=1159&c=3865&c=3866&c=1162&c=3867&c=3868&c=1166&c=1167&c=3869&c=1168&c=1169&c=1170&c=21&c=1171&c=1172&c=23&c=3870&c=1178&c=3871&c=1179&c=1180&c=3872&c=1183&c=1185&c=3873&c=3874&c=3875&c=3876&c=3877&c=3878&c=3879&c=1194&c=3880&c=3881&c=1203&c=3882&c=1206&c=3883&c=3884&c=3885&c=1210&c=3886&c=3887&c=1212&c=1213&c=3888&c=3889&c=3890&c=3891&c=3892&c=3893&c=3894&c=3895&c=3896&c=3897&c=3898&c=3899&c=3900&c=3901&c=3902&c=3903&c=8&c=3904&c=3905&c=3906&c=3907&c=3908&c=3909&c=3910&c=3911&c=3912&c=3913&c=3914&c=3915&c=3916&c=3917&c=3918&c=3919&c=3920&c=3921&c=3922&c=3923&c=3924&c=3925&c=3926&c=3927&c=3928&c=3929&c=3930&c=3931&c=1224&c=1228&c=1229&c=1231&c=3932&c=3933&c=3934&c=3935&c=3936&c=3937&c=3938&c=3939&c=3940&c=3941&c=3942&c=3943&c=3944&c=3945&c=3946&c=3947&c=3948&c=3949&c=3950&c=3951&c=3952&c=3953&c=3954&c=3955&c=3956&c=3957&c=3958&c=3959&c=3960&c=3961&c=3962&c=3963&c=3964&c=1266&c=3965&c=3966&c=3967&c=3968&c=3969&c=3970&c=3971&c=3972&c=3973&c=3974&c=3975&c=3976&c=3977&c=3978&c=3979&c=3980&c=3981&c=3982&c=3983&c=3984&c=3985&c=3986&c=3987&c=3988&c=3989&c=1320&c=3990&c=1322&c=3991&c=3992&c=3993&c=3994&c=3995&c=3996&c=3997&c=3998&c=3999&c=1335&c=36&c=4000&c=4001&c=1340&c=1341&c=1342&c=4002&c=4003&c=1348&c=1349&c=4004&c=4005&c=4006&c=4007&c=4008&c=1357&c=1358&c=1359&c=4009&c=1363&c=4010&c=4011&c=4012&c=4013&c=4014&c=1390&c=4015&c=4016&c=1394&c=4017&c=4018&c=4019&c=4020&c=4021&c=4022&c=1403&c=4023&c=1406&c=4024&c=4025&c=4026&c=4027&c=4028&c=4029&c=4030&c=4031&c=4032&c=1439&c=3&c=4033&c=4034&c=1449&c=1450&c=1451&c=4035&c=4036&c=1452&c=1453&c=4037&c=1455&c=4038&c=1456&c=1457&c=1458&c=4039&c=1460&c=4040&c=4041&c=4042&c=4043&c=1462&c=17&c=4044&c=4045&c=4046&c=18&c=4047&c=4048&c=4049&c=4050&c=4051&c=4052&c=4053&c=4054&c=4055&c=4056&c=4057&c=4058&c=4059&c=4060&c=4061&c=4062&c=1471&c=4063&c=4064&c=4065&c=4066&c=4067&c=1473&c=4068&c=4069&c=4070&c=4071&c=4072&c=4073&c=1482&c=1483&c=4074&c=4075&c=4076&c=4077&c=4078&c=4079&c=4080&c=4081&c=4082&c=4083&c=4084&c=4085&c=4086&c=4087&c=4088&c=4089&c=4090&c=4091&c=4092&c=4093&c=4094&c=4095&c=4096&c=4097&c=1489&c=4098&c=4099&c=4100&c=4101&c=4102&c=4103&c=1497&c=4104&c=4105&c=4106&c=4107&c=4108&c=4109&c=4110&c=4111&c=1501&c=1511&c=1512&c=4112&c=4113&c=4114&c=4115&c=4116&c=4117&c=4118&c=4119&c=4120&c=4121&c=1537&c=4122&c=4123&c=4124&c=4125&c=4126&c=4127&c=4128&c=4129&c=4130&c=4131&c=4132&c=4133&c=4134&c=4135&c=4136&c=4137&c=4138&c=1557&c=4139&c=4140&c=4141&c=4142&c=1563&c=4143&c=4144&c=4145&c=4146&c=4147&c=4148&c=4149&c=4150&c=1580&c=4151&c=4152&c=4153&c=4154&c=1583&c=4155&c=4156&c=4157&c=1588&c=1590&c=4158&c=4159&c=4160&c=4161&c=4162&c=4163&c=4164&c=4165&c=4166&c=4167&c=4168&c=4169&c=4170&c=4171&c=4172&c=4173&c=4174&c=4175&c=4176&c=4177&c=4178&c=4179&c=4180&c=4181&c=4182&c=4183&c=4184&c=4185&c=4186&c=1612&c=1613&c=4187&c=4188&c=4189&c=4190&c=4191&c=4192&c=4193&c=4194&c=4195&c=4196&c=4197&c=4198&c=4199&c=4200&c=4201&c=4202&c=4203&c=4204&c=4205&c=4206&c=4207&c=4208&c=4209&c=4210&c=4211&c=4212&c=4213&c=4214&c=1659&c=4215&c=4216&c=4217&c=4218&c=4219&c=4220&c=4221&c=4222&c=4223&c=4224&c=1673&c=4225&c=4226&c=4227&c=1680&c=1681&c=1684&c=4228&c=5&c=4229&c=4230&c=4231&c=6&c=7&c=1695&c=4232&c=4233&c=4234&c=4235&c=4236&c=30&c=4237&c=4238&c=4239&c=4240&c=4241&c=4242&c=4243&c=4244&c=4245&c=4246&c=1716&c=1718&c=1719&c=4247&c=4248&c=1723&c=4249&c=4250&c=4251&c=4252&c=4253&c=4254&c=4255&c=4256&c=4257&c=4258&c=4259&c=4260&c=4261&c=4262&c=4263&c=4264&c=1747&c=1749&c=1755&c=4265&c=4266&c=4267&c=4268&c=4269&c=4270&c=4271&c=4272&c=4273&c=4274&c=4275&c=4276&c=4277&c=4278&c=4279&c=4280&c=4281&c=4282&c=4283&c=4284&c=4285&c=4286&c=4287&c=4288&c=4289&c=4290&c=4291&c=4292&c=4293&c=4294&c=4295&c=4296&c=4297&c=4298&c=4299&c=4300&c=4301&c=4302&c=4303&c=4304&c=4305&c=4306&c=4307&c=4308&c=4309&c=4310&c=4311&c=4312&c=4313&c=4314&c=4315&c=4316&c=4317&c=4318&c=4319&c=4320&c=4321&c=4322&c=4323&c=4324&c=4325&c=4326&c=4327&c=4328&c=4329&c=4330&c=4331&c=4332&c=4333&c=4334&c=4335&c=4336&c=4337&c=4338&c=4339&c=4340&c=4341&c=4342&c=4343&c=4344&c=4345&c=4346&c=4347&c=4348&c=4349&c=4350&c=4351&c=4352&c=4353&c=4354&c=4355&c=4356&c=4357&c=4358&c=4359&c=4360&c=4361&c=4362&c=4363&c=4364&c=4365&c=4366&c=4367&c=4368&c=4369&c=4370&c=4371&c=4372&c=1916&c=1917&c=1918&c=4373&c=4374&c=4375&c=4376&c=4377&c=4378&c=4379&c=4380&c=4381&c=4382&c=4383&c=1938&c=4384&c=4385&c=1942&c=4386&c=4387&c=4388&c=4389&c=4390&c=4391&c=4392&c=4393&c=4394&c=4395&c=4396&c=4397&c=4398&c=4399&c=4400&c=4401&c=4402&c=4403&c=4404&c=4405&c=4406&c=4407&c=4408&c=4409&c=4410&c=4411&c=4412&c=1973&c=4413&c=4414&c=1984&c=4415&c=4416&c=1987&c=4417&c=4418&c=1989&c=4419&c=4420&c=4421&c=4422&c=4423&c=4424&c=4425&c=4426&c=1998&c=1999&c=2000&c=4427&c=4428&c=4429&c=4430&c=4431&c=4432&c=4433&c=4434&c=2013&c=4435&c=2016&c=4436&c=4437&c=4438&c=4439&c=4440&c=4441&c=2032&c=4442&c=2033&c=2034&c=2035&c=2036&c=4443&c=4444&c=4445&c=4446&c=2040&c=4447&c=4448&c=4449&c=4450&c=4451&c=4452&c=4453&c=4454&c=4455&c=32&c=4456&c=33&c=4457&c=4458&c=4459&c=24&c=4460&c=4461&c=4462&c=4463&c=4464&c=4465&c=4466&c=4467&c=4468&c=4469&c=2076&c=4470&c=4471&c=4472&c=4473&c=4474&c=4475&c=4476&c=4477&c=4478&c=4479&c=4480&c=4481&c=4482&c=4483&c=4484&c=4485&c=4486&c=4487&c=4488&c=4489&c=4490&c=4491&c=4492&c=4493&c=4494&c=4495&c=4496&c=4497&c=4498&c=4499&c=4500&c=4501&c=4502&c=4503&c=4504&c=4505&c=4506&c=4507&c=4508&c=4509&c=4510&c=4511&c=4512&c=4513&c=4514&c=4515&c=4516&c=4517&c=4518&c=4519&c=4520&c=4521&c=4522&c=4523&c=4524&c=4525&c=2093&c=2102&c=4526&c=4527&c=4528&c=4529&c=4530&c=4531&c=4532&c=4533&c=4534&c=4535&c=4536&c=2109&c=2110&c=2111&c=4537&c=2113&c=2114&c=2116&c=2117&c=2118&c=2119&c=2121&c=4538&c=2122&c=2130&c=2133&c=4539&c=2137&c=4540&c=4541&c=2141&c=2142&c=2143&c=4542&c=4543&c=4544&c=2148&c=4545&c=4546&c=4547&c=4548&c=4549&c=4550&c=4551&c=4552&c=4553&c=4554&c=4555&c=4556&c=4557&c=4558&c=4559&c=4560&c=4561&c=4562&c=4563&c=4564&c=4565&c=4566&c=4567&c=4568&c=4569&c=2178&c=2184&c=4570&c=4571&c=4572&c=4573&c=4574&c=4575&c=4576&c=4577&c=4578&c=4579&c=4580&c=4581&c=4582&c=2198&c=4583&c=4584&c=4585&c=4586&c=19&c=26&c=22&c=29&c=25&c=4587&c=4588&c=4589&c=2215&c=4590&c=4591&c=4592&c=4593&c=4594&c=4595&c=4596&c=4597'

The above returns curl: (56) Unexpected EOF. The connection is dropped in Chrome and Firefox, too. This only seems to occur with HTTP/2 - the connection succeeds with HTTP/1.1. I've tried raising all the various buffer parameters with no success. This issue can be reproduced using the current nginx:mainline Docker image.

Change History (11)

comment:1 by Valentin V. Bartenev, 6 years ago

Have you checked the error log? In most cases it contains the reason.

Also, it's not clear what "buffer parameters" you've tried to rise, but your request is bigger than the default http2_max_field_size value.

See the documentation:
http://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_field_size

Last edited 6 years ago by Valentin V. Bartenev (previous) (diff)

comment:2 by alubbock@…, 6 years ago

Raising both http2_max_field_size and http2_max_header_size fixes the issue. I hadn't realized HTTP/2 headers were handled separately in the configuration. Thanks for your help.

The only remaining issue is: shouldn't the server return HTTP 414, rather than dropping the connection? Any other concurrent requests over the same HTTP/2 connection are also dropped, which may be confusing for multi-threaded client applications.

comment:3 by Valentin V. Bartenev, 6 years ago

Since headers in HTTP/2 protocol are encoded using stateful compression algorithm, it's impossible to continue maintaining connection if there's any problem with handling headers in a request (e.g. limits are reached).

Note that nginx doesn't just drop the connection, but it sends a GOAWAY frame with ENHANCE_YOUR_CALM protocol error.

comment:4 by Maxim Dounin, 6 years ago

Resolution: invalid
Status: newclosed

comment:5 by Maxim Dounin, 5 years ago

See also #1749.
Other related tickets: #1420, #1387, #1027, #793.

comment:6 by Maxim Dounin, 5 years ago

See also #1800.

comment:7 by Maxim Dounin, 5 years ago

See also #1508.

comment:8 by Maxim Dounin, 5 years ago

See also #1802.

comment:9 by Maxim Dounin, 4 years ago

Se also #1866.

Version 0, edited 4 years ago by Maxim Dounin (next)

comment:10 by Maxim Dounin, 4 years ago

See also #1907.

comment:11 by Maxim Dounin, 4 years ago

See also #2005.

Note: See TracTickets for help on using tickets.