Opened 2 years ago

Closed 2 years ago

#2405 closed defect (invalid)

OCSP issue to specific responder

Reported by: dougherty.rr@… Owned by:
Priority: minor Milestone:
Component: nginx-module Version: 1.23.x
Keywords: ocsp, ngx_http_ssl_module Cc:
uname -a: Linux 51e1b00edc16 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.23.2
built by gcc 10.2.1 20210110 (Debian 10.2.1-6)
built with OpenSSL 1.1.1n 15 Mar 2022
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -ffile-prefix-map=/data/builder/debuild/nginx-1.23.2/debian/debuild-base/nginx-1.23.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

Description

I'm attempting to validate a client certificate with the OCSP responder http://eva.orc.com, however, it looks like nginx always gets a 404 when trying to validate a cert against this specific responder. I'm wondering if there is an issue in nginx that makes it incompatible with this responder. It's definitely possible I have an incorrect configuration, but I've been working on this for a while and have had no luck getting it working.

I'm using the latest container from dockerhub w/ debug on ([nginx-debug, '-g', 'daemon off;'])

Notes:

  • I can successfully validate a different client cert against an OCSP responder (ocsp.disa.mil). See logs below.
  • I can validate the client certificate when configuring a CRL in nginx using ssl_crl
  • I can validate the client cert against the responder using openssl commands (e.g. openssl ocsp -CAfile CAs.pem -issuer eca-issuer.pem -cert eca-cert.pem -text -url http://eva.orc.com)
  • I get a 200 response when curling the OCSP responder from the container
curl -is http://eva.orc.com
HTTP/1.0 200 OK
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Content-Security-Policy: default-src 'self' 'unsafe-inline'
Content-type: text/html

<HTML><HEAD>
<TITLE>Axway Validation Authority Server</TITLE>
</HEAD>
<BODY>
<H1>Welcome to the Axway Validation Authority Server Version 5.2</H1>
</BODY></HTML>

SSL Config:
I've tried many different implementations of this, but this is a common config that I've used.

server {
    listen 443 ssl;
    server_name localhost;

    ssl_certificate             /etc/nginx/localhost.crt;
    ssl_certificate_key         /etc/nginx/localhost.key;
    ssl_ciphers                 "ECDHE+AES:DHE+AES:@STRENGTH:+AES256:-3DES:!SHA1:!SHA256:!SHA384";
    ssl_protocols               TLSv1.2;
    ssl_verify_client           on;
    # pem format cert bundle
    ssl_client_certificate    /etc/nginx/CAs.crt;
    ssl_verify_depth            5;

    # ssl_trusted_certificate /etc/nginx/CAs.crt;
    ssl_ocsp                    on;
    resolver  127.0.0.11 8.8.8.8 8.8.4.4 valid=30s ipv6=off;
    

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

}

Logs:

### Unsuccessful (404 response) from eva.orc.com responder using ECA client cert

2022/10/27 18:26:38 [debug] 28#28: *7 ssl ocsp validate, certs:3
2022/10/27 18:26:38 [debug] 28#28: *7 ssl ocsp validate cert:0
2022/10/27 18:26:38 [debug] 28#28: *7 posix_memalign: 000055711C4BB420:2048 @16
2022/10/27 18:26:38 [debug] 28#28: *7 ssl ocsp request
2022/10/27 18:26:38 [debug] 28#28: *7 ssl ocsp request length 112, escape 2
2022/10/27 18:26:38 [debug] 28#28: malloc: 000055711C4B4920:224
2022/10/27 18:26:38 [debug] 28#28: resolve: "eva.orc.com"
2022/10/27 18:26:38 [debug] 28#28: malloc: 000055711C4EEE70:184
2022/10/27 18:26:38 [debug] 28#28: malloc: 000055711C530C70:11
2022/10/27 18:26:38 [debug] 28#28: malloc: 000055711C52F860:58
2022/10/27 18:26:38 [debug] 28#28: resolve: "eva.orc.com" A 52939
2022/10/27 18:26:38 [debug] 28#28: resolve: "eva.orc.com" AAAA 48210
2022/10/27 18:26:38 [debug] 28#28: UDP socket 13
2022/10/27 18:26:38 [debug] 28#28: connect to 127.0.0.11:53, fd:13 #8
2022/10/27 18:26:38 [debug] 28#28: epoll add event: fd:13 op:1 ev:80002001
2022/10/27 18:26:38 [debug] 28#28: send: fd:13 29 of 29
2022/10/27 18:26:38 [debug] 28#28: send: fd:13 29 of 29
2022/10/27 18:26:38 [debug] 28#28: malloc: 000055711C5318D0:96
2022/10/27 18:26:38 [debug] 28#28: event timer add: -1: 60000:1732000644
2022/10/27 18:26:38 [debug] 28#28: event timer add: -1: 5000:1731945644
2022/10/27 18:26:38 [debug] 28#28: timer delta: 540
2022/10/27 18:26:38 [debug] 28#28: worker cycle
2022/10/27 18:26:38 [debug] 28#28: epoll timer: 5000
2022/10/27 18:26:39 [debug] 28#28: epoll: fd:13 ev:0001 d:00007F82667E74C0
2022/10/27 18:26:39 [debug] 28#28: recv: fd:13 68 of 4096
2022/10/27 18:26:39 [debug] 28#28: resolver DNS response 52939 fl:8180 1/2/0/0
2022/10/27 18:26:39 [debug] 28#28: resolver DNS response qt:1 cl:1
2022/10/27 18:26:39 [debug] 28#28: malloc: 000055711C530CB0:12
2022/10/27 18:26:39 [debug] 28#28: resolver qs:eva.orc.com
2022/10/27 18:26:39 [debug] 28#28: resolver naddrs:1 cname:00007FFE42A04B79 ttl:900
2022/10/27 18:26:39 [debug] 28#28: recv: fd:13 -1 of 4096
2022/10/27 18:26:39 [debug] 28#28: recv() not ready (11: Resource temporarily unavailable)
2022/10/27 18:26:39 [debug] 28#28: timer delta: 227
2022/10/27 18:26:39 [debug] 28#28: worker cycle
2022/10/27 18:26:39 [debug] 28#28: epoll timer: 4773
2022/10/27 18:26:39 [debug] 28#28: epoll: fd:13 ev:0001 d:00007F82667E74C0
2022/10/27 18:26:39 [debug] 28#28: recv: fd:13 52 of 4096
2022/10/27 18:26:39 [debug] 28#28: resolver DNS response 48210 fl:8180 1/1/0/0
2022/10/27 18:26:39 [debug] 28#28: resolver DNS response qt:28 cl:1
2022/10/27 18:26:39 [debug] 28#28: malloc: 000055711C530CB0:12
2022/10/27 18:26:39 [debug] 28#28: resolver qs:eva.orc.com
2022/10/27 18:26:39 [debug] 28#28: resolver naddrs:0 cname:00007FFE42A04B79 ttl:900
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp resolve handler
2022/10/27 18:26:39 [debug] 28#28: *7 name was resolved to 208.16.159.137
2022/10/27 18:26:39 [debug] 28#28: resolve name done: 0
2022/10/27 18:26:39 [debug] 28#28: event timer del: -1: 1732000644
2022/10/27 18:26:39 [debug] 28#28: resolver expire
2022/10/27 18:26:39 [debug] 28#28: event timer del: -1: 1731945644
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp connect 0/1
2022/10/27 18:26:39 [debug] 28#28: *7 stream socket 15
2022/10/27 18:26:39 [debug] 28#28: *7 epoll add connection: fd:15 ev:80002005
2022/10/27 18:26:39 [debug] 28#28: *7 connect to 208.16.159.137:80, fd:15 #9
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp connect peer done
2022/10/27 18:26:39 [debug] 28#28: recv: fd:13 -1 of 4096
2022/10/27 18:26:39 [debug] 28#28: recv() not ready (11: Resource temporarily unavailable)
2022/10/27 18:26:39 [debug] 28#28: timer delta: 38
2022/10/27 18:26:39 [debug] 28#28: worker cycle
2022/10/27 18:26:39 [debug] 28#28: epoll timer: 59195
2022/10/27 18:26:39 [debug] 28#28: epoll: fd:15 ev:0004 d:00007F82667E75B0
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp write handler
2022/10/27 18:26:39 [debug] 28#28: *7 send: fd:15 153 of 153
2022/10/27 18:26:39 [debug] 28#28: timer delta: 162
2022/10/27 18:26:39 [debug] 28#28: worker cycle
2022/10/27 18:26:39 [debug] 28#28: epoll timer: 59033
2022/10/27 18:26:39 [debug] 28#28: epoll: fd:15 ev:2005 d:00007F82667E75B0
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp read handler
2022/10/27 18:26:39 [debug] 28#28: *7 malloc: 000055711C524FF0:16384
2022/10/27 18:26:39 [debug] 28#28: *7 recv: eof:1, avail:-1
2022/10/27 18:26:39 [debug] 28#28: *7 recv: fd:15 26 of 16384
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp process status line
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp status 404 "404 Not Found"
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp process headers
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp process body
2022/10/27 18:26:39 [debug] 28#28: *7 recv: eof:1, avail:0
2022/10/27 18:26:39 [debug] 28#28: *7 recv: fd:15 0 of 16358
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp process body
2022/10/27 18:26:39 [debug] 28#28: *7 ssl ocsp done
### Successful 200 response from ocsp.disa.mil responder using a different client cert

2022/10/31 13:52:42 [debug] 29#29: *7 ssl ocsp validate, certs:3
2022/10/31 13:52:42 [debug] 29#29: *7 ssl ocsp validate cert:0
2022/10/31 13:52:42 [debug] 29#29: *7 posix_memalign: 00005607E047D420:2048 @16
2022/10/31 13:52:42 [debug] 29#29: *7 ssl ocsp request
2022/10/31 13:52:42 [debug] 29#29: *7 ssl ocsp request length 96, escape 2
2022/10/31 13:52:42 [debug] 29#29: malloc: 00005607E0506AA0:224
2022/10/31 13:52:42 [debug] 29#29: resolve: "ocsp.disa.mil"
2022/10/31 13:52:42 [debug] 29#29: malloc: 00005607E04F2450:184
2022/10/31 13:52:42 [debug] 29#29: malloc: 00005607E047FBD0:13
2022/10/31 13:52:42 [debug] 29#29: malloc: 00005607E047F090:31
2022/10/31 13:52:42 [debug] 29#29: resolve: "ocsp.disa.mil" A 52497
2022/10/31 13:52:42 [debug] 29#29: UDP socket 13
2022/10/31 13:52:42 [debug] 29#29: connect to 127.0.0.11:53, fd:13 #8
2022/10/31 13:52:42 [debug] 29#29: epoll add event: fd:13 op:1 ev:80002001
2022/10/31 13:52:42 [debug] 29#29: send: fd:13 31 of 31
2022/10/31 13:52:42 [debug] 29#29: malloc: 00005607E054A900:96
2022/10/31 13:52:42 [debug] 29#29: event timer add: -1: 30000:266832982
2022/10/31 13:52:42 [debug] 29#29: event timer add: -1: 5000:266807982
2022/10/31 13:52:42 [debug] 29#29: timer delta: 5735
2022/10/31 13:52:42 [debug] 29#29: worker cycle
2022/10/31 13:52:42 [debug] 29#29: epoll timer: 5000
2022/10/31 13:52:42 [debug] 29#29: epoll: fd:13 ev:0001 d:00007FBF4714E4C0
2022/10/31 13:52:42 [debug] 29#29: recv: fd:13 86 of 4096
2022/10/31 13:52:42 [debug] 29#29: resolver DNS response 52497 fl:8180 1/3/0/0
2022/10/31 13:52:42 [debug] 29#29: resolver DNS response qt:1 cl:1
2022/10/31 13:52:42 [debug] 29#29: malloc: 00005607E0480380:14
2022/10/31 13:52:42 [debug] 29#29: resolver qs:ocsp.disa.mil
2022/10/31 13:52:42 [debug] 29#29: resolver naddrs:2 cname:00007FFF3DAFF85B ttl:10
2022/10/31 13:52:42 [debug] 29#29: malloc: 00005607E0480380:8
2022/10/31 13:52:42 [debug] 29#29: malloc: 00005607E04F23F0:80
2022/10/31 13:52:42 [debug] 29#29: malloc: 00005607E04F5480:224
2022/10/31 13:52:42 [debug] 29#29: *7 ssl ocsp resolve handler
2022/10/31 13:52:42 [debug] 29#29: *7 name was resolved to 156.112.111.142
2022/10/31 13:52:42 [debug] 29#29: *7 name was resolved to 156.112.101.142
2022/10/31 13:52:42 [debug] 29#29: resolve name done: 0
2022/10/31 13:52:42 [debug] 29#29: event timer del: -1: 266832982
2022/10/31 13:52:42 [debug] 29#29: resolver expire
2022/10/31 13:52:42 [debug] 29#29: event timer del: -1: 266807982
2022/10/31 13:52:42 [debug] 29#29: *7 ssl ocsp connect 0/2
2022/10/31 13:52:42 [debug] 29#29: *7 stream socket 15
2022/10/31 13:52:42 [debug] 29#29: *7 epoll add connection: fd:15 ev:80002005
2022/10/31 13:52:42 [debug] 29#29: *7 connect to 156.112.111.142:80, fd:15 #9
2022/10/31 13:52:42 [debug] 29#29: *7 ssl ocsp connect peer done
2022/10/31 13:52:42 [debug] 29#29: recv: fd:13 -1 of 4096
2022/10/31 13:52:42 [debug] 29#29: recv() not ready (11: Resource temporarily unavailable)
2022/10/31 13:52:42 [debug] 29#29: timer delta: 287
2022/10/31 13:52:42 [debug] 29#29: worker cycle
2022/10/31 13:52:42 [debug] 29#29: epoll timer: 53978
2022/10/31 13:52:43 [debug] 29#29: epoll: fd:15 ev:0004 d:00007FBF4714E5B0
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp write handler
2022/10/31 13:52:43 [debug] 29#29: *7 send: fd:15 139 of 139
2022/10/31 13:52:43 [debug] 29#29: timer delta: 151
2022/10/31 13:52:43 [debug] 29#29: worker cycle
2022/10/31 13:52:43 [debug] 29#29: epoll timer: 53827
2022/10/31 13:52:43 [debug] 29#29: epoll: fd:15 ev:0005 d:00007FBF4714E5B0
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp read handler
2022/10/31 13:52:43 [debug] 29#29: *7 malloc: 00005607E0540770:16384
2022/10/31 13:52:43 [debug] 29#29: *7 recv: eof:0, avail:-1
2022/10/31 13:52:43 [debug] 29#29: *7 recv: fd:15 238 of 16384
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp process status line
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp status 200 "200 "
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp process headers
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp header "X-Frame-Options: DENY"
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp header "X-Content-Type-Options: nosniff"
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp header "X-XSS-Protection: 1; mode=block"
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp header "Content-Type: application/ocsp-response"
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp header "Content-Length: 3561"
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp header "Date: Mon, 31 Oct 2022 13:41:25 GMT"
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp header "Connection: close"
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp header "Server: Web"
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp process body
2022/10/31 13:52:43 [debug] 29#29: *7 recv: eof:0, avail:0
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp dummy handler
2022/10/31 13:52:43 [debug] 29#29: timer delta: 151
2022/10/31 13:52:43 [debug] 29#29: worker cycle
2022/10/31 13:52:43 [debug] 29#29: epoll timer: 53676
2022/10/31 13:52:43 [debug] 29#29: epoll: fd:15 ev:0005 d:00007FBF4714E5B0
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp read handler
2022/10/31 13:52:43 [debug] 29#29: *7 recv: eof:0, avail:-1
2022/10/31 13:52:43 [debug] 29#29: *7 recv: fd:15 2920 of 16146
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp process body
2022/10/31 13:52:43 [debug] 29#29: *7 recv: eof:0, avail:0
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp dummy handler
2022/10/31 13:52:43 [debug] 29#29: timer delta: 1
2022/10/31 13:52:43 [debug] 29#29: worker cycle
2022/10/31 13:52:43 [debug] 29#29: epoll timer: 53675
2022/10/31 13:52:43 [debug] 29#29: epoll: fd:15 ev:0004 d:00007FBF4714E5B0
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp dummy handler
2022/10/31 13:52:43 [debug] 29#29: timer delta: 0
2022/10/31 13:52:43 [debug] 29#29: worker cycle
2022/10/31 13:52:43 [debug] 29#29: epoll timer: 53675
2022/10/31 13:52:43 [debug] 29#29: epoll: fd:15 ev:2005 d:00007FBF4714E5B0
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp read handler
2022/10/31 13:52:43 [debug] 29#29: *7 recv: eof:1, avail:-1
2022/10/31 13:52:43 [debug] 29#29: *7 recv: fd:15 641 of 13226
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp process body
2022/10/31 13:52:43 [debug] 29#29: *7 recv: eof:1, avail:0
2022/10/31 13:52:43 [debug] 29#29: *7 recv: fd:15 0 of 12585
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp process body
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp response, good, 3561
2022/10/31 13:52:43 [debug] 29#29: *7 ssl ocsp done

Change History (3)

comment:1 by Maxim Dounin, 2 years ago

Symptoms suggest that the OCSP responser you are trying to use simply does not support GET OCSP requests, and therefore always returns 404 to such requests. And, since nginx uses GET requests to ensure proper HTTP-level caching, it doesn't work.

Unfortunately, OCSP responders which only support POST requests appear in the wild, despite the fact that properly implemented OCSP responders are expected to support both GET and POST. Most notably, openssl ocsp ... only supports POST in both the client and the test server, which makes it appear that such POST-only OCSP responders do actually work, while in fact they are "only useful for test and demonstration purposes" as correctly noted in the OpenSSL's OCSP server documentation.

Not sure there are widely available tools which can be used to independently verify if the OCSP responder is able to respond to GET OCSP requests or not. On macOS, at least security verify-cert -R ocsp -c <cert> uses GET internally for requests smaller than 255 bytes (here), which basically applies to all requests. Another method might be to construct an OCSP request in DER format with openssl ocsp ... and manually create a GET OCSP request as per RFC 6960. Alternatively, you can capture OCSP requests as sent by nginx on the wire, check their syntax yourself, and try to replay them manually to the OCSP responder.

Another option might be to contact the certificate authority to ensure that the OCSP responder you are trying to use is actually expected to work with GET OCSP requests. The 404 response suggests it isn't, and needs fixing.

comment:2 by dougherty.rr@…, 2 years ago

Thanks for the reply. I reached out to the company that provides the client certs and they said that they do support OCSP get requests. I'll try some of your other suggestions to see if I can get any more info debug info.

comment:3 by Maxim Dounin, 2 years ago

Resolution: invalid
Status: newclosed

Feedback timeout. As previously suggested, from the symptoms provided this looks like an issue in the particular OCSP responder.

Note: See TracTickets for help on using tickets.