Opened 3 years ago

Closed 3 days ago

#1250 closed defect (fixed)

Connection reset with low http2_max_requests

Reported by: Lisio@… Owned by:
Priority: major Milestone:
Component: nginx-module Version: 1.11.x
Keywords: http2_max_requests Cc:
uname -a: Linux domain.name 4.9.21-mod-std-ipv6-64 #1 SMP Tue Apr 11 14:57:57 CEST 2017 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.11.13
built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4)
built with OpenSSL 1.0.2g 1 Mar 2016
TLS SNI support enabled
configure arguments: --prefix=/opt/nginx-1.11.13 --user=nginx --group=nginx --with-http_ssl_module --with-http_gzip_static_module --with-ipv6 --with-http_v2_module

Description

I have a page with ~400 image thumbs and when http2_max_requests is set low (even with default 1000) I get connection reset errors in any browser on any OS after some page reloads. If I set this parameter to 100 - I get error on each page and most of the images are not loaded.

Change History (8)

comment:1 by sunnybear@…, 3 years ago

Confirm this issue on slow connections (detect very rarely, on slow WiFi only)

comment:2 by Valentin V. Bartenev, 3 years ago

There are known issues in browsers: Chrome and Firefox.

comment:3 by Lisio@…, 3 years ago

Tried to find browser that works normally with current nginx but without success.

comment:4 by nh2@…, 2 years ago

I believe the browsers are not at fault.

Nginx is not sending the data in a way that allows the browsers to receive it.

When nginx reaches http2_max_requests, it sends GOAWAY and closes the socket using close() (observed in strace).
But packets from the other side can still be in flight.
When an in-flight packet arrives at the kernel of the machine running nginx, after close(), the kernel will reply with TCP RST.
When the browser side receives the RST, its kernel will discard any receive buffers, even for packets that it already ACKed.
These receive buffers may contain the GOAWAY (or the last headers/data after it in the same stream).
Thus the browser may never learn that the server sent GOAWAY.

This page from 2009 describes that in detail: https://blog.netherlabs.nl/articles/2009/01/18/the-ultimate-so_linger-page-or-why-is-my-tcp-not-reliable

As presented on https://bugzilla.mozilla.org/show_bug.cgi?id=1050329#c14, the only solution seems to be to not close() after sending GOAWAY, but to shutdown() instead (which is also the solution from the page linked above).


In line with that: The Chrome issue linked above is marked as fixed, nevertheless the problem continues to exist. The most relevant Chrome issue to this bug is probably this.

comment:5 by nh2@…, 2 years ago

A workaround right now seems to be to set http2_max_requests extremely large (e.g. to many millions; unfortunately it seems it can't be disabled completely).

I have asked on https://trac.nginx.org/nginx/ticket/1102#comment:4 whether this has a drawback.

comment:6 by Maxim Dounin, 4 weeks ago

See also #1544, which is related. Both problems should be solved once lingering close will be introduced in HTTP/2.

comment:7 by Ruslan Ermilov <ru@…>, 3 days ago

In 7673:c5840ca2063d/nginx:

HTTP/2: lingering close after GOAWAY.

After sending the GOAWAY frame, a connection is now closed using
the lingering close mechanism.

This allows for the reliable delivery of the GOAWAY frames, while
also fixing connection resets observed when http2_max_requests is
reached (ticket #1250), or with graceful shutdown (ticket #1544),
when some additional data from the client is received on a fully
closed connection.

For HTTP/2, the settings lingering_close, lingering_timeout, and
lingering_time are taken from the "server" level.

comment:8 by Ruslan Ermilov, 3 days ago

Resolution: fixed
Status: newclosed

HTTP/2 connections now support lingering close mechanism.

Note: See TracTickets for help on using tickets.