Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#1257 closed defect (invalid)

Some nginx TLS tests started failing with LibreSSL 2.5.3

Reported by: jirutka@… Owned by:
Priority: critical Milestone:
Component: nginx-core Version:
Keywords: libressl tls security Cc: ncopa@…
uname -a:
nginx -V: nginx version: nginx/1.10.3 built with LibreSSL 2.5.3 TLS SNI support enabled configure arguments: --prefix=/var/lib/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --pid-path=/run/nginx/nginx.pid --lock-path=/run/nginx/nginx.lock --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --with-perl_modules_path=/usr/lib/perl5/vendor_perl --user=nginx --group=nginx --with-threads --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_random_index_module --with-http_secure_link_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-http_realip_module --with-mail=dynamic --with-mail_ssl_module --with-stream=dynamic --with-stream_ssl_module

Description

After we updated LibreSSL from 2.4.4. to 2.5.3 in Alpine Linux, we have noticed that some TLS-related tests in nginx (both 1.10.3 and 1.12.0) started failing. Moreover, most of them fail because nginx accepted certificate that should be rejected! That’s pretty bad regression.

We’re not sure if the problem is in LibreSSL, nginx or nginx-tests, so reporting it to both.

People from VoidLinux? has reproduced this issue too, on glibc.

Test Summary Report
-------------------
./h2_ssl_verify_client.t             (Wstat: 256 Tests: 5 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
./mail_imap_ssl.t                    (Wstat: 512 Tests: 14 Failed: 2)
  Failed tests:  4, 10
  Non-zero exit status: 2
./proxy_bind_transparent.t           (Wstat: 512 Tests: 3 Failed: 2)
  Failed tests:  1-2
  Non-zero exit status: 2
./proxy_ssl_certificate.t            (Wstat: 256 Tests: 7 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
./ssl_crl.t                          (Wstat: 512 Tests: 5 Failed: 2)
  Failed tests:  2-3
  Non-zero exit status: 2
./ssl_verify_client.t                (Wstat: 512 Tests: 12 Failed: 2)
  Failed tests:  4-5
  Non-zero exit status: 2
./ssl_verify_depth.t                 (Wstat: 256 Tests: 4 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
./stream_proxy_ssl_certificate.t     (Wstat: 256 Tests: 7 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
./stream_ssl_verify_client.t         (Wstat: 512 Tests: 12 Failed: 2)
  Failed tests:  3, 5
  Non-zero exit status: 2
Files=290, Tests=3628, 300 wallclock secs ( 1.91 usr  0.46 sys + 54.82 cusr  9.07 csys = 66.26 CPU)

Complete log: http://tpaste.us/Ynw6

Attachments (1)

nginx-tests.log (25.1 KB) - added by jirutka@… 2 years ago.
nginx-tests log

Download all attachments as: .zip

Change History (10)

Changed 2 years ago by jirutka@…

nginx-tests log

comment:2 Changed 2 years ago by jirutka@…

VoidLinux? guys started investigating it more and according to them the problem lies in LibreSSL, not nginx.

comment:3 follow-up: Changed 2 years ago by jirutka@…

So, Duncaen from VoidLinux? has identified potential cause in LibreSSL code, commit https://github.com/libressl-portable/openbsd/commit/ddd98f8ea741a122952185a36c1396c14c2fda74. I’ve tried to build LibreSSL 2.5.3 with patch reverting that commit, rebuild nginx, run tests and… all pass! So it seems that this is really the cause.

However, it also seems that the problem is even on the nginx’s side, in function ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) (https://github.com/nginx/nginx/blob/branches/stable-1.10/src/event/ngx_event_openssl.c#L665-L706). It always returns 1, that seems to be wrong. It should return the given first argument (ok), that’s what null_callback(int ok, X509_STORE_CTX *e) (the default no-op callback) do, what other projects like OpenSMTPD use and what the OpenSSL documentation suggests. Why nginx always returns 1 here, i.e. saying that the cert is ok?


I’ve tried to change ngx_ssl_verify_callback to return the first given argument (ok) instead of 1 and build with unpatched LibreSSL 2.5.3.

--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -702,7 +702,7 @@
     }
 #endif

-    return 1;
+    return ok;
 }

Here are the results:

Test Summary Report
-------------------
./addition_buffered.t                (Wstat: 0 Tests: 3 Failed: 0)
  TODO passed:   1
./h2.t                               (Wstat: 0 Tests: 142 Failed: 0)
  TODO passed:   136-137
./h2_ssl_verify_client.t             (Wstat: 65280 Tests: 3 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 5 tests but ran 3.
./mail_imap_ssl.t                    (Wstat: 65280 Tests: 5 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 14 tests but ran 5.
./proxy_ssl_certificate.t            (Wstat: 256 Tests: 7 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
./ssl_crl.t                          (Wstat: 512 Tests: 5 Failed: 2)
  Failed tests:  2-3
  Non-zero exit status: 2
./ssl_verify_client.t                (Wstat: 65280 Tests: 5 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 12 tests but ran 5.
./ssl_verify_depth.t                 (Wstat: 256 Tests: 4 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
./stream_proxy_ssl_certificate.t     (Wstat: 256 Tests: 7 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
Files=290, Tests=3234, 303 wallclock secs ( 1.73 usr  0.54 sys + 54.39 cusr  8.60 csys = 65.26 CPU)
Result: FAIL

Complete log: http://tpaste.us/Dk8e

comment:4 in reply to: ↑ 3 Changed 2 years ago by pluknet

Replying to jirutka@…:

However, it also seems that the problem is even on the nginx’s side, in function ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) (https://github.com/nginx/nginx/blob/branches/stable-1.10/src/event/ngx_event_openssl.c#L665-L706). It always returns 1, that seems to be wrong.
..

That is incorrect. According to the documentation:

: The return value of verify_callback controls the strategy of the
: further verification process. If verify_callback returns 0, the
: verification process is immediately stopped with "verification failed"
: state. <..> If verify_callback returns 1, the verification process is continued.

This is what was broken in LibreSSL in the aforementioned change.

comment:5 Changed 2 years ago by pluknet

  • Resolution set to invalid
  • Status changed from new to closed

As previously noted, this is not a problem in nginx.

comment:6 Changed 2 years ago by jirutka@…

Please read also the rest of the documentation:

If verify_callback always returns 1, the TLS/SSL handshake will not be terminated with respect to verification failures and the connection will be established.

comment:7 follow-up: Changed 2 years ago by mdounin

Yes, and this is exactly what nginx wants to happen. It then uses SSL_get_verify_result() to check if the verification was successful or not, and acts appropriately - allows request to be handled, or returns an appropriate error. Unfortunately, this is broken in the current version of LibreSSL, and it does not seem to be possible to do anything with this on nginx side.

comment:8 in reply to: ↑ 7 Changed 2 years ago by jirutka@…

Replying to mdounin:

Thanks for clarification! We have discussed it on IRC and other guys agreed that you’re right. Also LibreSSL already responded on the issue:

Thanks for digging into the issue and reporting this. It seems that the change does indeed break the documented API - we're looking into solutions and will follow up soon.

comment:9 Changed 2 years ago by hdatma@…

Last edited 2 years ago by hdatma@… (previous) (diff)
Note: See TracTickets for help on using tickets.