Opened 5 years ago

Closed 5 years ago

#1805 closed defect (invalid)

Cannot reopen log files without restarting nginx

Reported by: ljelinek-niccz@… Owned by:
Priority: minor Milestone:
Component: nginx-core Version: 1.10.x
Keywords: log, Cc:
uname -a: Linux proxy 4.4.0-150-generic #176-Ubuntu SMP Wed May 29 18:56:26 UTC 2019 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.10.3
built with OpenSSL 1.1.0f 25 May 2017 (running with OpenSSL 1.1.0j 20 Nov 2018)
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fdebug-prefix-map=/build/nginx-0TiIP5/nginx-1.10.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_xslt_module=dynamic --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module --add-dynamic-module=/build/nginx-0TiIP5/nginx-1.10.3/debian/modules/nginx-auth-pam --add-dynamic-module=/build/nginx-0TiIP5/nginx-1.10.3/debian/modules/nginx-dav-ext-module --add-dynamic-module=/build/nginx-0TiIP5/nginx-1.10.3/debian/modules/nginx-echo --add-dynamic-module=/build/nginx-0TiIP5/nginx-1.10.3/debian/modules/nginx-upstream-fair --add-dynamic-module=/build/nginx-0TiIP5/nginx-1.10.3/debian/modules/ngx_http_substitutions_filter_module

Description

Reopening log files after log rotation silently fails. There is no error reported but the old log files remain open instead of closing it and opening the new ones.

Steps to reproduce:

  1. Install nginx from Debian 9 or Ubuntu 16.04 packages.
  2. Let nginx to work some time in real operation to create some log records.
  3. Run logrotate.

Expected result:
The old log files are closed and the new ones are open.

Current result:
The old log files remain open (nginx writes into them) and the new ones remain intact.

Attempts to reopen log files using 'nginx -s reopen' or 'kill -USR1 cat /run/nginx.pid' have the same result. They don't work. The only way how to force reopening the log files is 'systemctl restart nginx' or similar restart commands.

Information for Debian 9: in special text fields (uname -a, nginx -V)

Information for Ubuntu 16.04:

uname -a
Linux hk 4.4.0-154-generic #181-Ubuntu SMP Tue Jun 25 05:29:03 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

nginx -V
nginx version: nginx/1.10.3 (Ubuntu)
built with OpenSSL 1.0.2g 1 Mar 2016
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-stream --with-stream_ssl_module --with-mail --with-mail_ssl_module --with-threads

Change History (3)

comment:1 by Maxim Dounin, 5 years ago

Reopening log files may fail if new log files cannot be opened for writing by worker processes. If this happens, nginx will log something like:

2019/07/08 18:23:17 [notice] 48190#100071: reopening logs
2019/07/08 18:23:17 [emerg] 48190#100071: open() "/path/to/log" failed (13: Permission denied)

to the error log set at global level (/var/log/nginx/error.log by default in your builds).

To further debug what goes on in your case, configure global error log to use notice level. Once done, make sure you can see reopening logs line as logged at the notice level. If you see this line, this indicate that that USR1 signal is delivered to nginx and nginx should either reopen logs or log an error. If you don't see this line, this likely indicate that USR1 signal is not delivered - either you are using wrong PID file to send signals, or its contents were somehow damaged - check PID file contents and compare it to ps output on your system.

comment:2 by ljelinek-niccz@…, 5 years ago

Thank you for your help. The problem was caused by insufficient privileges on the /var/log/nginx directory. This issue may be closed.

comment:3 by Maxim Dounin, 5 years ago

Resolution: invalid
Status: newclosed
Note: See TracTickets for help on using tickets.