Opened 3 years ago

Closed 3 years ago

#1340 closed defect (worksforme)

Incorrect TLS session resumption for SNI-enabled upstream.

Reported by: oleg.guba@… Owned by:
Priority: minor Milestone:
Component: nginx-core Version: 1.11.x
Keywords: SNI upstream Cc:
uname -a: Linux xxx 4.8.0-46-generic #49~16.04.1-Ubuntu SMP Fri Mar 31 14:51:03 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.11.13 (1.11.13.XXX, rbtz, 2017-05-17 20:25:18 UTC (release))
built with OpenSSL 1.0.2j 26 Sep 2016
TLS SNI support enabled
configure arguments: --prefix=/usr/local --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/client_temp --http-proxy-temp-path=/var/lib/nginx/proxy_temp --http-fastcgi-temp-path=/var/lib/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/lib/nginx/uwsgi_temp --http-scgi-temp-path=/var/lib/nginx/scgi_temp --with-http_stub_status_module --with-http_ssl_module --with-http_gzip_static_module --with-http_realip_module --with-stream --with-http_v2_module --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro' --build='1.11.13.XXX, rbtz, 2017-05-17 20:25:18 UTC (release)' --builddir=objs-release --with-openssl=opaque-build/openssl-1.0.2j --with-zlib=opaque-build/zlib-1.2.11 --with-pcre=opaque-build/pcre-8.40 --with-pcre-opt=-DSUPPORT_UCP --with-pcre-jit --with-openssl-opt='no-seed no-bf no-cast no-md2 no-md4 no-ripemd no-krb5 no-jpake no-capieng no-idea no-rc5 no-mdc2 no-ssl2 threads enable-tlsext no-shared enable-ec_nistp_64_gcc_128' --add-module=opaque-build/ngx_devel_kit-0.3.0 --add-module=opaque-build/headers-more-nginx-0.32 --add-module=opaque-build/ngx_http_ip_tos_filter_module --add-module=modules/dbx_dials_module --add-module=modules/lua-nginx-module --add-module=modules/nginx_upstream_check_module --add-module=modules/ngx_http_streaming_filter_module --add-module=modules/ngx_http_upstream_dynamic_module --add-module=modules/ngx_brotli

Description

I use configuration like this:

http {
    upstream upstream.domain.com {
        keepalive 128;
        server 1.1.1.1:443 max_fails=0;
    }
    server {
        listen 1.2.3.4:443 ssl;
        ssl_certificate x.domain.com.crt;
        ssl_certificate_key x.domain.com.key;

        server_name x.domain.com;
        location / {
            proxy_pass https://upstream.domain.com;
        }
    }
    server {
        listen 1.2.3.4:443 ssl;
        ssl_certificate y.domain.com.crt;
        ssl_certificate_key x.domain.com.key;
        server_name y.domain.com;
        location / {
            proxy_pass https://upstream.domain.com;
        }
    }
}

So nginx serves two different https servers with different certificates on one IP, both of them use same upstream.
Upstream 1.1.1.1:443 is also nginx-powered, has the same configuration with SNI (except the upstream host).

Using this configuration some (not all) requests to x.domain.com or y.domain.com fail with:

[error] 94934#0: *1388162830 upstream SSL certificate does not match "x.domain.com" while SSL handshaking to upstream, client: 10.48.0.154, server: x.domain.com, request: "POST /create HTTP/1.1", upstream: "https://1.1.1.1:443/create", host: "x.domain.com"

It fails here: https://github.com/nginx/nginx/blob/9edd64fcd842870ea004664288cadc344c33f0bd/src/event/ngx_event_openssl.c#L3177
I found that this error only happens when nginx tries to reuse ssl session to upstream.
I added some additional logging and found that sometimes reusing ssl session nginx gets incorrect cert (for upstream with server_name y.domain.com instead of x.domain.com).
Disabling ssl session reuse for upstream (proxy_ssl_session_reuse off) solves the problem.
I expect there is some issues with ssl session cache for SNI-enabled upstreams.

Change History (2)

comment:1 by Maxim Dounin, 3 years ago

SSL sessions are cached in the context of the upstream{} block (or an implicit upstream when using an IP address or a DNS name). That is, a configuration like this:

proxy_ssl_verify on;
proxy_ssl_server_name on;

location /x/ {
    proxy_pass https://x.example.com;
}

location /y/ {
    proxy_pass https://y.example.com;
}

is expected to work fine without any problems, and the error you see should never happen.

On the other hand, if you use the same upstream for multiple SNI names, the error is expected. That is, if you use something like:

proxy_ssl_verify on;
proxy_ssl_server_name on;

location /x/ {
    proxy_pass https://127.0.0.1;
    proxy_ssl_name x.example.com;
}

location /y/ {
    proxy_pass https://127.0.0.1;
    proxy_ssl_name y.example.com;
}

it will expectedly fail with session reuse, because sessions to https://127.0.0.1 will be cached regardless of SNI name and/or other SSL parameters used.

Given the above, I suspect that your proxying configuration is what causes the error you see. Please show your proxying configuration to confirm.

comment:2 by Maxim Dounin, 3 years ago

Resolution: worksforme
Status: newclosed

Feedback timeout.

Note: See TracTickets for help on using tickets.