Allow using OpenSSL 3.0 "provider" API instead of deprecated "engine" API

[nickrbogdanov@…]

I would like to use hardware encryption to protect my webserver's TLS privkey from theft. My server has a TPM 2.0 chip which supports this, but it's unnecessarily difficult to configure it in nginx. I'm proposing a change to make the process more user-friendly.

For background: old versions of OpenSSL supported HSMs and hardware offloading through the ENGINE API. In nginx this is enabled through the ssl_engine directive: hxxps://github.com/Infineon/optiga-tpm-cheatsheet#nginx--curl

(Please s/hxxps/https/ due to Trac spam filter)

In OpenSSL 3.0 the authors introduced a "Provider API" which is intended to replace the old ENGINE API. It works in a similar way. When using the CLI to sign or create TPM-backed keys, you add -provider tpm2 -provider base to the arguments: hxxps://github.com/Infineon/optiga-tpm-cheatsheet#pem-encoded-key-object-2

I'm running Ubuntu 22.04 LTS on this webserver. This Linux distribution has a tpm2-openssl package ( hxxps://github.com/tpm2-software/tpm2-openssl ) which conforms to the new OpenSSL 3.0 Provider API. The CLI examples on the optiga-tpm-cheatsheet work right out of the box, with no extra configuration. This makes it quick and easy to set up hardware-backed keys, just by installing a single package. For instance, anyone running this distro on an x86 Linux PC can do:

sudo apt install tpm2-openssl
openssl genpkey -provider tpm2 -algorithm EC -pkeyopt ec_paramgen_curve:P-384 -out testkey.priv
echo "test" | openssl pkeyutl -provider tpm2 -provider base -digest sha256 -inkey testkey.priv -sign -rawin -hexdump

But unfortunately, at this time nginx only supports the ENGINE API, not the Provider API. In order to use hardware backed keys with nginx, users would need to compile, install, and configure the legacy tpm2tss ENGINE implementation, and keep it up to date themselves without help from the Debian/Ubuntu package maintainers.

I believe that with a small tweak to nginx, it would be possible for users to specify e.g.

ssl_provider tpm2,base

to tell OpenSSL 3.x to use the tpm2-openssl Provider to support hardware-backed private keys in nginx.

[Maxim Dounin]

Note that it should be possible to load OpenSSL providers using the configuration file, see ​README-PROVIDERS.md in OpenSSL sources for an example.

Support for loading providers directly from nginx configuration might still be useful from simplicity point of view, though it does not look like something required to use tpm2-openssl provider.

[nickrbogdanov@…]

What I found when enabling the tpm2+base providers globally in openssl.cnf, is that applications could utilize the encrypted TSS2 privkeys but they lost the ability to sign using standard non-TPM-backed privkeys. Thus forcing me to choose between "all TPM2" or "all default," systemwide.

I think that if nginx called SSL_CTX_config(ctx, "nginx") then I could add an nginx-specific provider configuration to openssl.cnf and that would largely avoid this downside? But maybe there's a better way to do this without an nginx code change.

[Maxim Dounin]

It is certainly possible to provide an nginx-specific OpenSSL configuration file via the OPENSSL_CONF environment variable.

[Maxim Dounin]

See also #2507.

[Maxim Dounin <mdounin@…>]

In 9136:85abf534cead/nginx:

SSL: provided "nginx" appname when loading OpenSSL configs.

Following OpenSSL 0.9.8f, OpenSSL tries to load application-specific
configuration section first, and then falls back to the "openssl_conf"
default section if application-specific section is not found, by using
CONF_modules_load_file(CONF_MFLAGS_DEFAULT_SECTION). Therefore this
change is not expected to introduce any compatibility issues with existing
configurations. It does, however, make it easier to configure specific
OpenSSL settings for nginx in system-wide OpenSSL configuration
(ticket #2449).

Instead of checking OPENSSL_VERSION_NUMBER when using the OPENSSL_init_ssl()
interface, the code now tests for OPENSSL_INIT_LOAD_CONFIG to be defined and
true, and also explicitly excludes LibreSSL. This ensures that this interface
is not used with BoringSSL and LibreSSL, which do not provide additional
library initialization settings, notably the OPENSSL_INIT_set_config_appname()
call.

[Maxim Dounin]

For the record, the above commit makes it possible to provide nginx-specific configuration in system-wide openssl.cnf. For example, following the ​PROVIDERS.md example configuration:

openssl_conf = openssl_init

[openssl_init]
providers = provider_sect

[provider_sect]
default = default_sect
legacy = legacy_sect

[default_sect]
activate = 1

[legacy_sect]
activate = 1

It is now possible to provide nginx-specific configuration, by using nginx instead of openssl_conf:

nginx = openssl_init

[openssl_init]
providers = provider_sect

[provider_sect]
default = default_sect
legacy = legacy_sect

[default_sect]
activate = 1

[legacy_sect]
activate = 1

In particular, this can be used to configure providers specifically for nginx via system-wide OpenSSL configuration file, without using nginx-specific OpenSSL configuration file and OPENSSL_CONF.

[omayevskiy@…]

It is possible to load providers via openssl.cnf

For example

[openssl_init]
providers = provider_sect

# List of providers to load
[provider_sect]
default = default_sect
pkcs11 = pkcs11_sect

[default_sect]
activate = 1

[pkcs11_sect]
module = /usr/local/lib/ossl-modules/pkcs11.so
pkcs11-module-path = /usr/local/lib/softhsm/libsofthsm2.so
activate = 1

But unfortunately, nginx does not support selecting a specific provider if using the ​ssl_certificate_key directive.

The value provider:name:id cannot be specified instead of the file, which would load a secret key with a specified id from the OpenSSL provider name.