Opened 9 years ago
Last modified 9 years ago
#796 reopened defect
nginx.pid is removed during reload if pid path is changed in nginx.conf but points to the same file through a symlink
Reported by: | Owned by: | ||
---|---|---|---|
Priority: | minor | Milestone: | |
Component: | nginx-core | Version: | 1.9.x |
Keywords: | Cc: | ||
uname -a: | Linux patrickxps 4.2.0-10-generic #12-Ubuntu SMP Tue Sep 15 19:43:01 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux | ||
nginx -V: |
nginx version: nginx/1.9.3 (Ubuntu)
built with OpenSSL 1.0.2d 9 Jul 2015 TLS SNI support enabled configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_spdy_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module |
Description
On recent debian systems and derivatives like Ubuntu, /var/run is a symlink to /run:
# ls -l /var/run lrwxrwxrwx 1 root root 4 Jan 21 2015 /var/run -> /run
On such systems, if the path to the pid file is changed from /var/run/nginx.pid to /run/nginx.pid in nginx.conf, effectively pointing to the same file on the filesystem, then when the configuration is reloaded the new /run/nginx.pid file is created then destroyed when the old /var/run/nginx.pid is unlinked.
Following are the steps to reproduce this issue.
At first, the nginx service is running normally:
# grep nginx.pid /etc/nginx/nginx.conf pid /var/run/nginx.pid; # cat /var/run/nginx.pid 1129 # ps -ef|grep nginx root 1129 1 0 Sep21 ? 00:00:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; www-data 4290 1129 0 07:49 ? 00:00:00 nginx: worker process www-data 4291 1129 0 07:49 ? 00:00:00 nginx: worker process www-data 4292 1129 0 07:49 ? 00:00:00 nginx: worker process www-data 4293 1129 0 07:49 ? 00:00:00 nginx: worker process
Simply reloading the configuration works fine:
# kill -HUP 1129 # ps -ef|grep nginx root 1129 1 0 Sep21 ? 00:00:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; www-data 5171 1129 0 07:50 ? 00:00:00 nginx: worker process www-data 5172 1129 0 07:50 ? 00:00:00 nginx: worker process www-data 5173 1129 0 07:50 ? 00:00:00 nginx: worker process www-data 5174 1129 0 07:50 ? 00:00:00 nginx: worker process # cat /var/run/nginx.pid 1129
Changing the pid path from /var/run/nginx.pid to /run/nginx.pid then reloading the configuration exhibits the issue:
# sed -i 's/pid \/var\/run\/nginx.pid/pid \/run\/nginx.pid/' /etc/nginx/nginx.conf # grep nginx.pid /etc/nginx/nginx.conf pid /run/nginx.pid; # kill -HUP 1129 # ps -ef|grep nginx root 1129 1 0 Sep21 ? 00:00:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on; www-data 5950 1129 0 07:51 ? 00:00:00 nginx: worker process www-data 5951 1129 0 07:51 ? 00:00:00 nginx: worker process www-data 5952 1129 0 07:51 ? 00:00:00 nginx: worker process www-data 5953 1129 0 07:51 ? 00:00:00 nginx: worker process # cat /var/run/nginx.pid cat: /var/run/nginx.pid: No such file or directory
Using strace, this behaviour is explained because the previous path is unlinked after the new file is created:
open("/var/run/nginx.pid", O_RDWR|O_CREAT|O_TRUNC, 0644) = 7 pwrite(7, "1129\n", 5, 0) = 5 close(7) = 0 unlink("/run/nginx.pid") = 0
Issue first encountered on Debian Wheezy with nginx 1.6.2 from backports, then reproduced on Ubuntu with nginx 1.9.3.
Change History (7)
comment:1 by , 9 years ago
comment:2 by , 9 years ago
I don't think anything can be done with this: it's the file name that nginx checks to find out if the file is the same or not. If it's not the same, nginx removes previous pid file it created much like on normal exit, to ensure no stray files are left. Using a symlink confuses this test, and this results in the unlink() shown. As far as I understand, symlink also won't survive nginx restart, which makes it more or less useless.
If you think things can be improved - suggestions are welcome.
comment:4 by , 9 years ago
Hi mdounin,
sorry I did not register to get email for changes regarding this issue,
so I did not receive the notice it was requiring feedback.
I guess the issue could be fixed by checking if the new and old filenames point to the same PID file, resolving symlink in the process and not deleting the PID file in that case.
Trying to restart nginx after the pid was destroyed will fail because the stopping part will not work, the old process will still be bound to the configured ports and the new process will fail to bind it.
comment:5 by , 9 years ago
Resolution: | wontfix |
---|---|
Status: | closed → reopened |
comment:6 by , 9 years ago
I have posted code which fixes this issue to nginx-devel: http://mailman.nginx.org/pipermail/nginx-devel/2016-February/007961.html
I'm keenly interested in feedback.
comment:7 by , 9 years ago
Hi mdounin,
I never got a response on the dev mailing list regarding my posted patch. I'm curious if this will get fixed soon.
thanks!
Oops, the above strace is from a subsequent test, the other way around (replacing /run/nginx.pid by /var/run/nginx.pid).
Here is the correct strace: