#1449 closed defect (worksforme)
ocsp failed, nginx failed to establish new connections
| Reported by: | https://stackoverflow.com/users/1100117/higuita | Owned by: | |
|---|---|---|---|
| Priority: | major | Milestone: | |
| Component: | other | Version: | 1.10.x |
| Keywords: | Cc: | ||
| uname -a: | Linux nginxlb--i-0ffcc4148076db4c9 4.9.0-4-amd64 #1 SMP Debian 4.9.51-1 (2017-09-28) x86_64 GNU/Linux | ||
| nginx -V: |
nginx version: nginx/1.10.3
built with OpenSSL 1.1.0f 25 May 2017 TLS SNI support enabled configure arguments: --with-cc-opt='-g -O2 -fdebug-prefix-map=/build/nginx-2tpxfc/nginx-1.10.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_xslt_module=dynamic --with-stream=dynamic --with-stream_ssl_module --with-mail=dynamic --with-mail_ssl_module --add-dynamic-module=/build/nginx-2tpxfc/nginx-1.10.3/debian/modules/nginx-auth-pam --add-dynamic-module=/build/nginx-2tpxfc/nginx-1.10.3/debian/modules/nginx-dav-ext-module --add-dynamic-module=/build/nginx-2tpxfc/nginx-1.10.3/debian/modules/nginx-echo --add-dynamic-module=/build/nginx-2tpxfc/nginx-1.10.3/debian/modules/nginx-upstream-fair --add-dynamic-module=/build/nginx-2tpxfc/nginx-1.10.3/debian/modules/ngx_http_substitutions_filter_module |
||
Description
Using the mozilla config generator, https://mozilla.github.io/server-side-tls/ssl-config-generator/ , i have the ssl stampling.
Today the server stop receiving connections and in the logs i got this:
2017/12/16 13:31:33 [error] 2069#2069: ocsp.comodoca.com could not be resolved (110: Operation timed out) while requesting certificate status, responder: ocsp.comodoca.com
2017/12/16 13:36:16 [error] 2069#2069: unexpected response for ocsp.comodoca.com
2017/12/16 13:36:16 [error] 2069#2069: unexpected response for ocsp.comodoca.com
2017/12/16 13:36:16 [error] 2069#2069: unexpected response for ocsp.comodoca.com
2017/12/16 13:36:48 [error] 2069#2069: ocsp.comodoca.com could not be resolved (110: Operation timed out) while requesting certificate status, responder: ocsp.comodoca.com
2017/12/16 13:36:57 [error] 2069#2069: unexpected response for ocsp.comodoca.com
2017/12/16 13:36:58 [error] 2069#2069: unexpected response for ocsp.comodoca.com
2017/12/16 13:36:58 [error] 2069#2069: unexpected response for ocsp.comodoca.com
2017/12/16 13:43:03 [error] 2069#2069: ocsp.comodoca.com could not be resolved (110: Operation timed out) while requesting certificate status, responder: ocsp.comodoca.com
2017/12/16 13:55:32 [error] 2069#2069: ocsp.comodoca.com could not be resolved (110: Operation timed out) while requesting certificate status, responder: ocsp.comodoca.com
2017/12/16 13:55:43 [error] 2069#2069: unexpected response for ocsp.comodoca.com
2017/12/16 13:55:43 [error] 2069#2069: unexpected response for ocsp.comodoca.com
restarting the nginx was enough to solve this... but of course, nginx should not lock up with the ocsp fails
Change History (2)
comment:1 by , 7 years ago
| Resolution: | → worksforme |
|---|---|
| Status: | new → closed |
comment:2 by , 7 years ago
Thanks for the feedback!
Notice that i could connect to other services at that time on the same server and i already saw that error in the past without any nginx problem (so proving that this alone should not be a problem), so i suspected a non-usual path/response that locked the nginx ssl part.
i was able to see that nginx got stuck "slowly", it could still response, but after some time no more new connections were accepted. I would say that keepalive connections where working, but something was locking the creating of new ssl connections.
we have another nginx in the same network and that one was working fine during the same time.
If it happens again, i will try to do a strace and a gdb to the nginx to try to give more hints what is happening
Messages suggests that nginx was not able to resole the OCSP responder name - from the logs it looks like DNS responses were coming after nginx given up waiting for them. These and other OCSP-related errors will not prevent nginx from working though, it will continue handling connections without OCSP stapling.
The root cause of the name resolution problems though - either non-working DNS server as seen from logs, or may be some network problems which caused DNS to be unresponsive - might be the real reason of the observed connectivity issues. You may want to dig further to understand what actually happened with your server.