Opened 3 years ago

Closed 3 years ago

#1419 closed defect (fixed)

The "worker process is shutting down" is running all the time, How should I do?

Reported by: lazyvislee@… Owned by:
Priority: minor Milestone: 1.13
Component: other Version: 1.12.x
Keywords: worker_shutdown_timeout Cc: gcc, 4.4.7
uname -a: Linux test 2.6.32-279.el6.x86_64
nginx -V: nginx version: nginx/1.12.1
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC)
built with OpenSSL 1.0.1e-fips 11 Feb 2013
TLS SNI support enabled

Description

The nginx is http proxy. when i use upgrade websocket and send heartbeat per 5s(client_body_timeout 6s;) the directivesworker_shutdown_timeout is invalid, the "worker process is shutting down" produced by nginx -s reload is running all the time.

How should I do?

Change History (8)

comment:1 by Maxim Dounin, 3 years ago

Resolution: invalid
Status: newclosed

For questions, please see support options available. Please avoid using Trac for questions, it is to report bugs.

comment:2 by lazyvislee@…, 3 years ago

I think it might be a bug.
I add a patch, The directivesworker_shutdown_timeout can take effect.
Is there a problem? Thx!!!

--- ngx_cycle.c	2017-07-11 21:24:07.000000000 +0800
+++ ../../../patch/ngx_cycle.c	2017-11-04 22:11:52.000000000 +0800
@@ -1379,6 +1379,9 @@
         c[i].close = 1;
         c[i].error = 1;
 
+        // the handler maybe ngx_http_upstream_read_request_handler
+        c[i].read->timedout = 1;
+
         c[i].read->handler(c[i].read);
     }
 }

comment:3 by Maxim Dounin, 3 years ago

No, the timedout flag is not set intentionally, because it is not a connection timeout, but rather a forced connection close. Handlers are expected to look at the c->close flag. Additionally, the c->error flag is set as a fallback, so handlers which doesn't handle the c->close flag directly will still close a connection once they'll try to output anything. That is, while the ngx_http_upstream_read_request_handler does not handle the `c->close' flag directly, the connection is still expected to be closed once it'll advance to the next state and, for example, will try sending something to the client.

comment:4 by Maxim Dounin, 3 years ago

Resolution: invalid
Status: closedreopened

It looks like the problem is that the worker_shutdown_timeout directive does not work for you in a specific situation, not that nginx rejects it as an invalid. See here for a suggested patch.

comment:5 by lazyvislee@…, 3 years ago

I will try it. Thank you!

comment:6 by lazyvislee@…, 3 years ago

The patch is effective. Will be merge into the mainline version?

comment:7 by Maxim Dounin <mdounin@…>, 3 years ago

In 7156:9c29644f6d03/nginx:

Fixed worker_shutdown_timeout in various cases.

The ngx_http_upstream_process_upgraded() did not handle c->close request,
and upgraded connections do not use the write filter. As a result,
worker_shutdown_timeout did not affect upgraded connections (ticket #1419).
Fix is to handle c->close in the ngx_http_request_handler() function, thus
covering most of the possible cases in http handling.

Additionally, mail proxying did not handle neither c->close nor c->error,
and thus worker_shutdown_timeout did not work for mail connections. Fix is
to add c->close handling to ngx_mail_proxy_handler().

Also, added explicit handling of c->close to stream proxy,
ngx_stream_proxy_process_connection(). This improves worker_shutdown_timeout
handling in stream, it will no longer wait for some data being transferred
in a connection before closing it, and will also provide appropriate
logging at the "info" level.

comment:8 by Maxim Dounin, 3 years ago

Resolution: fixed
Status: reopenedclosed

Fix committed, thanks.

Note: See TracTickets for help on using tickets.