Nginx failing to ask for PEM SSL key password

[lakarjail@…]

While starting/restarting nginx with "service nginx start", no password is asked on the terminal and nginx fails to start.

By checking journalctl, I receive the following error :

nov. 17  ... systemd[1]: Failed to reset devices.list on /system.slice/nginx.service: No 
nov. 17 ... nginx[1441]: Enter PEM pass phrase:
nov. 17  ... nginx[1441]: nginx: [emerg] SSL_CTX_use_PrivateKey_file("/etc/nginx/ssl/mykeycert") failed (SSL: error:0906406D:PEM routines:PEM_def_callback:problems getting password error:0906A
nov. 17 ... nginx[1441]: nginx: configuration file /etc/nginx/nginx.conf test failed
nov. 17 ... systemd[1]: nginx.service: control process exited, code=exited status=1
nov. 17 ... systemd[1]: Failed to start A high performance web server and a revers

The nginx server fail to start because no password is provided, however no password is required when starting it with service or /etc/init.d/.

Please note that :

1. nginx server starts correctly in command line (#nginx ), not using service. SSL configuration (like file locations and permissions seems therefore correct). Password is -that way- asked on terminal.
2. when doing the same SSL configuration with Apache2, the password is well required when starting/restarting Apache2 server with "service apache2 start".

How could I fix it please ? Is there a way to compare nginx and apache2 service files to understand why no password is asked when starting nginx, but actually is when starting apache2 ?

Thank you in advance for your help.

[lakarjail@…]

I failed to give the proper -V output :

nginx version: nginx/1.6.2
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt=-Wl,-z,relro --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_spdy_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module --add-module=/tmp/buildd/nginx-1.6.2/debian/modules/nginx-auth-pam --add-module=/tmp/buildd/nginx-1.6.2/debian/modules/nginx-dav-ext-module --add-module=/tmp/buildd/nginx-1.6.2/debian/modules/nginx-echo --add-module=/tmp/buildd/nginx-1.6.2/debian/modules/nginx-upstream-fair --add-module=/tmp/buildd/nginx-1.6.2/debian/modules/ngx_http_substitutions_filter_module

[Maxim Dounin]

The problem here is that a) your SSL keys are password-protected, so you have to enter a password, and b) systemd doesn't allow you to do so. Given the Apache2 behaviour, it's probably possible to teach systemd to allow nginx to ask for a password, but it won't really help to solve the problem, as nginx, e.g., may need to re-read SSL keys during configuration reload.

Trivial solution would be to remove a password from SSL keys used, using the openssl rsa command. Alternative one would be to use the ​ssl_password_file directive, as available in nginx 1.7.3+.

If you have further questions, please use ​mailing lists. Trac is to report and track bugs, and it's better to avoid using it as a support channel.

[lakarjail@…]

Maybe I mis-explained my issue. I am sorry if I posted my question at the wrong channel. For the record, I was told, on IRC nginx channel, to report it here.

1) I am not about to remove password of a cert key, since it's usually a bad security practise (considering the server get compromised, the cert will have to be revoked, etc.).
On top of that, as explained, I never had problems on Apache2 using a password protected key Cert file. When I run Apache service, password is well asked. I can not consider the solution of removing the password, when other solutions work properly.
I also checked ssl_password_file proposal. Storing the password in that way would set the security system as if no password was set on the key cert file. Therefore, I can't follow that solution neither.

2) What I fail to understand, if it is a bug, or a feature is the following : Nginx, when run as command line asks me for my cert key password and runs correctly. Why this behaviour can't be applied on a service ?
The command:

# nginx

Asks for a password, runs webserver Nginx correctly. However :

# service nginx start

doesn't, password is not asked on terminal, producing the journalctl above mentionned. Why this difference of response ? Why an Apache2-like (that works in both situation) mechanism can't be introduced with Nginx ?

Thank you in advance for your answer.

[Ilyas Bakirov]

Replying to lakarjail@…:

2) What I fail to understand, if it is a bug, or a feature is the following : Nginx, when run as command line asks me for my cert key password and runs correctly. Why this behaviour can't be applied on a service ?
The command:

# nginx

Asks for a password, runs webserver Nginx correctly. However :

# service nginx start

doesn't, password is not asked on terminal, producing the journalctl above mentionned. Why this difference of response ? Why an Apache2-like (that works in both situation) mechanism can't be introduced with Nginx ?

Consult/support and take look at your linux OS distribution documentation about nginx package, how service nginx is starting via service command. So here is difference in starting via systemd init system which may indicate pointed to another instance or may lack additional args/environment vars/perms

[Maxim Dounin]

Replying to lakarjail@…:

1) I am not about to remove password of a cert key, since it's usually a bad security practise (considering the server get compromised, the cert will have to be revoked, etc.).

If your server will be compromised, you'll have to revoke the cert anyway, as it's mostly trivial to extract an unencrypted key from a server memory. The only real case when encrypted keys can help is backup scenario - where keys can be accidentally backed up to an unsecure storage, and this won't result in an immediate keys compromise if keys are encrypted (and similar case happens when a server compromise requires a reboot).

On top of that, as explained, I never had problems on Apache2 using a password protected key Cert file. When I run Apache service, password is well asked. I can not consider the solution of removing the password, when other solutions work properly.

As previously explained, you can consider teaching systemd to allow nginx to ask passwords. But this won't really solve the problem during configuration reloads. And this is out of scope of nginx, as nginx doesn't provide any systemd service files at all, consider asking your package maintainer instead.

I also checked ssl_password_file proposal. Storing the password in that way would set the security system as if no password was set on the key cert file. Therefore, I can't -as well- follow that solution.

The problem is that entering a password is as well no different from storing unencrypted keys (unless you'll consider server reboots and/or backups). On the other hand, ssl_password_file allows you to don't store passwords anywhere (but use a named pipe instead), and this is no different from entering a password even if you'll consider backups/reboots. See ​http://nginx.org/r/ssl_password_file for an example.

2) What I fail to understand, if it is a bug, or a feature is the following : Nginx, when run as command line asks me for my cert key password and runs correctly. Why this behaviour can't be applied on a service ?

See above, no systemd service files are provided by nginx. Consider asking your package maintainer instead.

[2i@…]

I have written a script as a workaround which creates as decrypted copy of the key, starts nginx and removes the decypted key again.

First I created a copy of my tls server key to not have to change the server configuration:

cp /usr/local/nginx/conf/server.key /usr/local/nginx/conf/drubn.de.key.orig

Then this script located at /root/start_nginx.sh for example:

#!/bin/bash
orig_key_file=/usr/local/nginx/conf/server.key.orig
key_file=/usr/local/nginx/conf/server.key
trap "rm -f $key_file" EXIT && \
touch $key_file && \
chmod g=,o= $key_file && \
openssl rsa -in $orig_key_file -out $key_file && \
systemctl start nginx && \
rm -f $key_file

It’s not perfect, but I hope it helps. A better approach would probably be to use named pipes with the mentioned ssl_password_file directive to reduce the chance that the decrypted key or the password is being written to disk. But ssl_password_file exists only in nginx version ≥ 1.7.3.

In my opinion, it should not be an absolute law that developers of a software may not talk to package maintainers. What is the aim of the nginx contributors: to produce source code or to provide people a fast and easily configurable web server?

Ok, maybe this is just a bug tracker for the source code…

Yours faithfully!