Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#556 closed defect (wontfix)

OCSP stapling not working with comodo PositiveSSL

Reported by: Frederik Schwan Owned by:
Priority: major Milestone:
Component: nginx-module Version: 1.5.x
Keywords: OCSP SSL Cc:
uname -a: Linux heimdall 3.14.2-1-ARCH #1 SMP PREEMPT Sun Apr 27 11:28:44 CEST 2014 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.6.0
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --conf-path=/etc/nginx/nginx.conf --sbin-path=/usr/bin/nginx --pid-path=/run/nginx.pid --lock-path=/run/lock/nginx.lock --user=http --group=http --http-log-path=/var/log/nginx/access.log --error-log-path=stderr --http-client-body-temp-path=/var/lib/nginx/client-body --http-proxy-temp-path=/var/lib/nginx/proxy --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-imap --with-imap_ssl_module --with-ipv6 --with-pcre-jit --with-file-aio --with-http_dav_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_realip_module --with-http_spdy_module --with-http_ssl_module --with-http_stub_status_module --with-http_addition_module --with-http_degradation_module --with-http_flv_module --with-http_mp4_module --with-http_secure_link_module --with-http_sub_module

Description

When using OCSP stapling

resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
ssl_stapling on;
ssl_stapling_verify off;
ssl_trusted_certificate /etc/ssl/private/chain.comodo.pem;
ssl_dhparam /etc/nginx/dh_2048.pem;

I get the following log entry:

Mai 04 19:12:35 heimdall nginx[4573]: 2014/05/04 19:12:35 [error] 4576#0: OCSP_basic_verify() failed (SSL: error:27069076:OCSP routines:OCSP_basic_verify:signer certificate not found) while requesting certificate status, responder: ocsp.comodoca.com

So I decided to do some trial and error with openssl. The only way the certificate has been proved correct is to use the "-VAfile" option. Using CA, CAfile does not work. Seems to me that the way comodo signs its OCSP answer is not very usual. They don't ship the signing certificate with the answer and don't mark the cert as a OCSP signing certificate. Lastly they even don't use their CA Cert, but an intermediate certificate (the issuer) to sign.

Since the "-VAfile" option works I suggest to let nginx invoke openssl with this option.

RootCA and intermediates attached as well as a OCSP answer generated with openssl (OpenSSL 1.0.1g 7 Apr 2014).

Attachments (1)

resp.der (472 bytes ) - added by Frederik Schwan 10 years ago.
OCSP response

Download all attachments as: .zip

Change History (11)

by Frederik Schwan, 10 years ago

Attachment: resp.der added

OCSP response

comment:1 by Frederik Schwan, 10 years ago

Upload button does not let me upload the cert file. So I put it here:

-----BEGIN CERTIFICATE-----
MIIGCDCCA/CgAwIBAgIQKy5u6tl1NmwUim7bo3yMBzANBgkqhkiG9w0BAQwFADCB
hTELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4G
A1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxKzApBgNV
BAMTIkNPTU9ETyBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTQwMjEy
MDAwMDAwWhcNMjkwMjExMjM1OTU5WjCBkDELMAkGA1UEBhMCR0IxGzAZBgNVBAgT
EkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMR
Q09NT0RPIENBIExpbWl0ZWQxNjA0BgNVBAMTLUNPTU9ETyBSU0EgRG9tYWluIFZh
bGlkYXRpb24gU2VjdXJlIFNlcnZlciBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEP
ADCCAQoCggEBAI7CAhnhoFmk6zg1jSz9AdDTScBkxwtiBUUWOqigwAwCfx3M28Sh
bXcDow+G+eMGnD4LgYqbSRutA776S9uMIO3Vzl5ljj4Nr0zCsLdFXlIvNN5IJGS0
Qa4Al/e+Z96e0HqnU4A7fK31llVvl0cKfIWLIpeNs4TgllfQcBhglo/uLQeTnaG6
ytHNe+nEKpooIZFNb5JPJaXyejXdJtxGpdCsWTWM/06RQ1A/WZMebFEh7lgUq/51
UHg+TLAchhP6a5i84DuUHoVS3AOTJBhuyydRReZw3iVDpA3hSqXttn7IzW3uLh0n
c13cRTCAquOyQQuvvUSH2rnlG51/ruWFgqUCAwEAAaOCAWUwggFhMB8GA1UdIwQY
MBaAFLuvfgI9+qbxPISOre44mOzZMjLUMB0GA1UdDgQWBBSQr2o6lFoL2JDqElZz
30O0Oija5zAOBgNVHQ8BAf8EBAMCAYYwEgYDVR0TAQH/BAgwBgEB/wIBADAdBgNV
HSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwGwYDVR0gBBQwEjAGBgRVHSAAMAgG
BmeBDAECATBMBgNVHR8ERTBDMEGgP6A9hjtodHRwOi8vY3JsLmNvbW9kb2NhLmNv
bS9DT01PRE9SU0FDZXJ0aWZpY2F0aW9uQXV0aG9yaXR5LmNybDBxBggrBgEFBQcB
AQRlMGMwOwYIKwYBBQUHMAKGL2h0dHA6Ly9jcnQuY29tb2RvY2EuY29tL0NPTU9E
T1JTQUFkZFRydXN0Q0EuY3J0MCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5jb21v
ZG9jYS5jb20wDQYJKoZIhvcNAQEMBQADggIBAE4rdk+SHGI2ibp3wScF9BzWRJ2p
mj6q1WZmAT7qSeaiNbz69t2Vjpk1mA42GHWx3d1Qcnyu3HeIzg/3kCDKo2cuH1Z/
e+FE6kKVxF0NAVBGFfKBiVlsit2M8RKhjTpCipj4SzR7JzsItG8kO3KdY3RYPBps
P0/HEZrIqPW1N+8QRcZs2eBelSaz662jue5/DJpmNXMyYE7l3YphLG5SEXdoltMY
dVEVABt0iN3hxzgEQyjpFv3ZBdRdRydg1vs4O2xyopT4Qhrf7W8GjEXCBgCq5Ojc
2bXhc3js9iPc0d1sjhqPpepUfJa3w/5Vjo1JXvxku88+vZbrac2/4EjxYoIQ5QxG
V/Iz2tDIY+3GH5QFlkoakdH368+PUq4NCNk+qKBR6cGHdNXJ93SrLlP7u3r7l+L4
HyaPs9Kg4DdbKDsx5Q5XLVq4rXmsXiBmGqW5prU5wfWYQ//u+aen/e7KJD2AFsQX
j4rBYKEMrltDR5FL1ZoXX/nUh8HCjLfn4g8wGTeGrODcQgPmlKidrv0PJFGUzpII
0fxQ8ANAe4hZ7Q7drNJ3gjTcBpUC2JD5Leo31Rpg0Gcg19hCC0Wvgmje3WYkN5Ap
lBlGGSW4gNfL1IYoakRwJiNiqZ+Gb7+6kHDSVneFeO/qJakXzlByjAA6quPbYzSf
+AZxAeKCINT+b72x
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIFdDCCBFygAwIBAgIQJ2buVutJ846r13Ci/ITeIjANBgkqhkiG9w0BAQwFADBv
MQswCQYDVQQGEwJTRTEUMBIGA1UEChMLQWRkVHJ1c3QgQUIxJjAkBgNVBAsTHUFk
ZFRydXN0IEV4dGVybmFsIFRUUCBOZXR3b3JrMSIwIAYDVQQDExlBZGRUcnVzdCBF
eHRlcm5hbCBDQSBSb290MB4XDTAwMDUzMDEwNDgzOFoXDTIwMDUzMDEwNDgzOFow
gYUxCzAJBgNVBAYTAkdCMRswGQYDVQQIExJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAO
BgNVBAcTB1NhbGZvcmQxGjAYBgNVBAoTEUNPTU9ETyBDQSBMaW1pdGVkMSswKQYD
VQQDEyJDT01PRE8gUlNBIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkq
hkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAkehUktIKVrGsDSTdxc9EZ3SZKzejfSNw
AHG8U9/E+ioSj0t/EFa9n3Byt2F/yUsPF6c947AEYe7/EZfH9IY+Cvo+XPmT5jR6
2RRr55yzhaCCenavcZDX7P0N+pxs+t+wgvQUfvm+xKYvT3+Zf7X8Z0NyvQwA1onr
ayzT7Y+YHBSrfuXjbvzYqOSSJNpDa2K4Vf3qwbxstovzDo2a5JtsaZn4eEgwRdWt
4Q08RWD8MpZRJ7xnw8outmvqRsfHIKCxH2XeSAi6pE6p8oNGN4Tr6MyBSENnTnIq
m1y9TBsoilwie7SrmNnu4FGDwwlGTm0+mfqVF9p8M1dBPI1R7Qu2XK8sYxrfV8g/
vOldxJuvRZnio1oktLqpVj3Pb6r/SVi+8Kj/9Lit6Tf7urj0Czr56ENCHonYhMsT
8dm74YlguIwoVqwUHZwK53Hrzw7dPamWoUi9PPevtQ0iTMARgexWO/bTouJbt7IE
IlKVgJNp6I5MZfGRAy1wdALqi2cVKWlSArvX31BqVUa/oKMoYX9w0MOiqiwhqkfO
KJwGRXa/ghgntNWutMtQ5mv0TIZxMOmm3xaG4Nj/QN370EKIf6MzOi5cHkERgWPO
GHFrK+ymircxXDpqR+DDeVnWIBqv8mqYqnK8V0rSS527EPywTEHl7R09XiidnMy/
s1Hap0flhFMCAwEAAaOB9DCB8TAfBgNVHSMEGDAWgBStvZh6NLQm9/rEJlTvA73g
JMtUGjAdBgNVHQ4EFgQUu69+Aj36pvE8hI6t7jiY7NkyMtQwDgYDVR0PAQH/BAQD
AgGGMA8GA1UdEwEB/wQFMAMBAf8wEQYDVR0gBAowCDAGBgRVHSAAMEQGA1UdHwQ9
MDswOaA3oDWGM2h0dHA6Ly9jcmwudXNlcnRydXN0LmNvbS9BZGRUcnVzdEV4dGVy
bmFsQ0FSb290LmNybDA1BggrBgEFBQcBAQQpMCcwJQYIKwYBBQUHMAGGGWh0dHA6
Ly9vY3NwLnVzZXJ0cnVzdC5jb20wDQYJKoZIhvcNAQEMBQADggEBAGS/g/FfmoXQ
zbihKVcN6Fr30ek+8nYEbvFScLsePP9NDXRqzIGCJdPDoCpdTPW6i6FtxFQJdcfj
Jw5dhHk3QBN39bSsHNA7qxcS1u80GH4r6XnTq1dFDK8o+tDb5VCViLvfhVdpfZLY
Uspzgb8c8+a4bmYRBbMelC1/kZWSWfFMzqORcUx8Rww7Cxn2obFshj5cqsQugsv5
B5a6SE2Q8pTIqXOi6wZ7I53eovNNVZ96YUWYGGjHXkBrI/V5eu+MtWuLt29G9Hvx
PUsE2JOAWVrgQSQdso8VYFhH2+9uRv0V9dlfmrPb2LjkQLPNlzmuhbsdjrzch5vR
pu/xO28QOG8=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIENjCCAx6gAwIBAgIBATANBgkqhkiG9w0BAQUFADBvMQswCQYDVQQGEwJTRTEU
MBIGA1UEChMLQWRkVHJ1c3QgQUIxJjAkBgNVBAsTHUFkZFRydXN0IEV4dGVybmFs
IFRUUCBOZXR3b3JrMSIwIAYDVQQDExlBZGRUcnVzdCBFeHRlcm5hbCBDQSBSb290
MB4XDTAwMDUzMDEwNDgzOFoXDTIwMDUzMDEwNDgzOFowbzELMAkGA1UEBhMCU0Ux
FDASBgNVBAoTC0FkZFRydXN0IEFCMSYwJAYDVQQLEx1BZGRUcnVzdCBFeHRlcm5h
bCBUVFAgTmV0d29yazEiMCAGA1UEAxMZQWRkVHJ1c3QgRXh0ZXJuYWwgQ0EgUm9v
dDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALf3GjPm8gAELTngTlvt
H7xsD821+iO2zt6bETOXpClMfZOfvUq8k+0DGuOPz+VtUFrWlymUWoCwSXrbLpX9
uMq/NzgtHj6RQa1wVsfwTz/oMp50ysiQVOnGXw94nZpAPA6sYapeFI+eh6FqUNzX
mk6vBbOmcZSccbNQYArHE504B4YCqOmoaSYYkKtMsE8jqzpPhNjfzp/haW+710LX
a0Tkx63ubUFfclpxCDezeWWkWaCUN/cALw3CknLa0Dhy2xSoRcRdKn23tNbE7qzN
E0S3ySvdQwAl+mG5aWpYIxG3pzOPVnVZ9c0p10a3CitlttNCbxWyuHv77+ldU9U0
WicCAwEAAaOB3DCB2TAdBgNVHQ4EFgQUrb2YejS0Jvf6xCZU7wO94CTLVBowCwYD
VR0PBAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wgZkGA1UdIwSBkTCBjoAUrb2YejS0
Jvf6xCZU7wO94CTLVBqhc6RxMG8xCzAJBgNVBAYTAlNFMRQwEgYDVQQKEwtBZGRU
cnVzdCBBQjEmMCQGA1UECxMdQWRkVHJ1c3QgRXh0ZXJuYWwgVFRQIE5ldHdvcmsx
IjAgBgNVBAMTGUFkZFRydXN0IEV4dGVybmFsIENBIFJvb3SCAQEwDQYJKoZIhvcN
AQEFBQADggEBALCb4IUlwtYj4g+WBpKdQZic2YR5gdkeWxQHIzZlj7DYd7usQWxH
YINRsPkyPef89iYTx4AWpb9a/IfPeHmJIZriTAcKhjW88t5RxNKWt9x+Tu5w/Rw5
6wwCURQtjr0W4MHfRnXnJK3s9EK0hZNwEGe6nQY1ShjTK3rMUUKhemPR5ruhxSvC
Nr4TDea9Y355e6cJDUCrat2PisP29owaQgVR1EX1n6diIWgVIEM8med8vSTYqZEX
c4g/VhsxOBi0cQ+azcgOno4uG+GMmIPLHzHxREzGBHNJdmAPx/i9F4BrLunMTA5a
mnkPIAou1Z5jJh5VkpTYghdae9C8x49OhgQ=
-----END CERTIFICATE-----

comment:2 by Frederik Schwan, 10 years ago

Patch provided here solves problem for openssl:
http://rt.openssl.org/Ticket/Display.html?id=2206

Patch: http://rt.openssl.org/Ticket/Attachment/26247/12756/implicit_non_delegated_ocsp_100beta5.patch

Since nginx has it's own logic to provide the arguments for OCSP_basic_verify in line 593 of ngx_event_openssl_stapling.c the patch need to be ported from openssl to nginx.

comment:3 by Frederik Schwan, 10 years ago

Ivestigated futher and made this patch:

--- aaa/src/event/ngx_event_openssl_stapling.c	2014-05-04 23:09:30.000000000 +0200
+++ bbb/src/event/ngx_event_openssl_stapling.c	2014-05-04 23:09:11.000000000 +0200
@@ -533,7 +533,7 @@
     size_t                 len;
     ngx_str_t              response;
     X509_STORE            *store;
-    STACK_OF(X509)        *chain;
+    STACK_OF(X509)        *chain, *issuer_st;
     OCSP_CERTID           *id;
     OCSP_RESPONSE         *ocsp;
     OCSP_BASICRESP        *basic;
@@ -553,6 +553,8 @@
 
     len = ctx->response->last - ctx->response->pos;
     p = ctx->response->pos;
+    issuer_st = sk_X509_new_null();
+    sk_X509_push(issuer_st, staple->issuer);
 
     ocsp = d2i_OCSP_RESPONSE(NULL, &p, len);
     if (ocsp == NULL) {
@@ -589,14 +591,17 @@
 #else
     chain = staple->ssl_ctx->extra_certs;
 #endif
-
+    
     if (OCSP_basic_verify(basic, chain, store,
-                          staple->verify ? OCSP_TRUSTOTHER : OCSP_NOVERIFY)
-        != 1)
-    {
-        ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0,
+                          staple->verify ? OCSP_TRUSTOTHER : OCSP_NOVERIFY) <= 0){
+        if(issuer_st && OCSP_basic_verify(basic, issuer_st, store, OCSP_TRUSTOTHER) > 0){
+            ERR_clear_error();
+        }
+        else {
+            ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0,
                       "OCSP_basic_verify() failed");
-        goto error;
+            goto error;
+        }
     }
 
     id = OCSP_cert_to_id(NULL, ctx->cert, ctx->issuer);

Checks issuer separately if check of chain fails. Works fine for me.
Thanks to Rob Stradling for providing the corresponding patch for OpenSSL.

Last edited 10 years ago by Frederik Schwan (previous) (diff)

comment:4 by Maxim Dounin, 10 years ago

Resolution: wontfix
Status: newclosed

It is believed that the underlying problem is in the OpenSSL library code, not in nginx. It should not fail OCSP response verification as long as the issuer certificate is in the store provided. Nevertheless, trivial workaround is to put issuer certificate into the certificate file as described in Configuring HTTPS servers instead of putting it into a trusted certificate store.

comment:5 by Frederik Schwan, 10 years ago

Putting issuer cert in certificate file is not a workaround - does not work.
Agree, that problem is the underlying OpenSSL lib. Will look into that.
Thanks

comment:6 by Maxim Dounin, 10 years ago

The workaround works fine here, I've tested it before I suggested this as a workaround. You may also find from the code that certificate chain is passed to the OCSP_basic_verify() as the "certs" argument, so it's basically identical to what you are doing with your patch. You may want to try again if it doesn't work for you.

comment:7 by Frederik Schwan, 10 years ago

Just tried it. Using cert-chain with issuer leads to "signer certificate not found"-error. If I try to manually check the OCSP response I have to use -VAfile issuer.crt. But without that option, only specifiying the whole trust chain (in which the issuer actually is) the ocsp response seems to be not valid.

I don't think that the OpenSSL patch I mentioned is the right way to fix this - it's only a workaround. There is something wrong in the OpenSSL routine that searches for the issuer in the chain.

BTW: Tried a RapidSSL cert. It's working fine with ocsp (it is not using the delegated model).

comment:8 by Maxim Dounin, 10 years ago

The workaround is not for OpenSSL command line interface, it's for nginx. Put a chain into the file you use in the ssl_certificate directive, as documented at http://nginx.org/en/docs/http/configuring_https_servers.html#chains. If it doesn't work for you, you may want to provide more details.

comment:9 by Frederik Schwan, 10 years ago

As I already wrote twice: I've tried to add the issuer cert to the nginx chain in many ways and also as described in the manpage (http://nginx.org/en/docs/http/configuring_https_servers.html#chains).

You find the chain, a working OpenSSL patch (workaround) and a working nginx patch (workaround) above.

You can break it down into two OpenSSL functions that are causing the problem:
ocsp_vfy.c - static int ocsp_find_signer
ocsp_vfy.c - static X509 *ocsp_find_signer_sk

The first one is called in ocsp_vfy.c at line 82. We get the error of this bug in line 85, since no issuer is found.
The strange thing: We find an issuer when STACK_OF(x509) *certs contains ONLY the issuer and nothing else....

comment:10 by Maxim Dounin, 10 years ago

The "strange thing" suggests that OpenSSL thinks there is no issuer certificate in the chain you are trying to use. This may be, e.g., if the pem file with the chain is corrupted, and e.g. there is a missing or extra char in the "-----BEGIN CERTIFICATE-----" line of the issuer certificate.

Note: See TracTickets for help on using tickets.