BoringSSL nginx, worker process XXXXX exited on signal 11

[Alex Storn]

Randomly crash nginx workers, used BoringSSL last version today. The statement took here ​https://calomel.org/nginx.html (build Nginx with Google's BoringSSL)

40k errors

[alert] 23307#0: worker process XXXXX exited on signal 11

This patch does not solve the problems workers crash

• ​http://trac.nginx.org/nginx/ticket/235

The crash is caused by SSL_CTX_get_ex_data() returning NULL, which is then dereferenced without a check. Since this check is absent in trunk, I suppose the bug is reproducible there, too.

worker_processes 4;
worker_priority 15;
pcre_jit on;
error_log  /var/log/nginx-error.log crit;
worker_rlimit_nofile 20000;
events {
  worker_connections 19000;
  use kqueue;
}

http {
 # Timeouts: do not keep connections open longer then necessary to reduce
 # resource usage and deny Slowloris type attacks. Slow attacks are related to
 # pausing in between packets.
  client_body_timeout      2s; # maximum time between packets the client can pause when sending nginx any data
  client_header_timeout    2s; # maximum time the client has to send the entire header to nginx
  keepalive_timeout       28s; # timeout which a single keep-alive client connection will stay open
  send_timeout            10s; # maximum time between packets nginx is allowed to pause when sending the client data
  spdy_keepalive_timeout 128s; # inactivity timeout after which the SPDY connection is closed
  spdy_recv_timeout        2s; # timeout if nginx is currently expecting data from the client but nothing arrives

 # general options for FreeBSD on ZFS
  aio                        on; # asynchronous file input/output, fast with ZFS, make sure sendfile=off
  charset                 utf-8; # adds the line "Content-Type" into response-header, same as "source_charset"
  default_type            application/octet-stream;
 #directio                  off; # zfs does not support direct i/o because of the ARC and L2ARC
  disable_symlinks           on; # disable symlinks to avoid malicious symlinks out of the document root
  etag                      off; # disables "ETag" response header so clients use Cache-Control header only
  gzip                      off; # disable on the fly gzip compression, only use gzip_static to reduce latency
  gzip_http_version         1.0; # serve gzipped content to all clients including HTTP/1.0 and greater
 # gzip_static            always; # precompress content (gzip -9) with an external script found on this page below
 #gzip_vary                  on; # send response header "Vary: Accept-Encoding". SPDY ignores Vary header
  gzip_proxied              any; # allows compressed responses for any request even from proxies
  ignore_invalid_headers     on;
  include                    mime.types;
  keepalive_requests         20; # number of keep alive requests per connection, does not affect SPDY
  keepalive_disable        none; # allow all browsers to use keepalive connections
  lingering_time              2; # maximum time during which nginx will process additional data from the client
  lingering_timeout           2; # maximum waiting time for more client data to arrive
  max_ranges                  1; # allow a single range header for resumed downloads and to stop large range header DoS attacks
  merge_slashes              on; # compression of two or more adjacent slashes in a URI into a single slash "//" into "/"
  msie_padding              off;
 #open_file_cache          max=128 inactive=4h; # cache is not be needed if ZFS ARC size is sufficient
 #open_file_cache_errors     on;                # since ARC delivery is faster then the cache lookups
 #open_file_cache_min_uses    1;
 #open_file_cache_valid      3h;
  output_buffers         1 256K; # sendfile=off so set to the total size of all objects on an average page
 #postpone_output          1460; # before sending data response, collect at least one packet's payload (MSS) of data
 #read_ahead                  0; # no forced read ahead, let ZFS handle I/O calls as zfs is efficient
  recursive_error_pages      on;
  reset_timedout_connection  on; # reset timed out connections freeing ram and resources
  sendfile                  off; # off for FreeBSD and ZFS to avoid redundant data caching
  server_tokens             off; # no nginx version number in error pages
 #server_name_in_redirect   off; # if off, nginx will use the requested Host header
  source_charset          utf-8; # same value as "charset"
  spdy_headers_comp           1; # SPDY gzip header compression to at least one(1) (default 0)
  spdy_max_concurrent_streams 20; #SPDY maximum parallel client requests (default 100)
  tcp_nodelay                on; # disable the Nagle buffering algorithm, used for keepalive only
  tcp_nopush                off; # sendfile=off so tcp_nopush can not be used

  ..
}

[Alex Storn]

When building using clang35 problem not reproducible, only base-system clang33.

ngx_int_t
ngx_ssl_handshake(ngx_connection_t *c)
{
    int        n, sslerr;
    ngx_err_t  err;

    ngx_ssl_clear_error(c->log);

 >>>>>   n = SSL_do_handshake(c->ssl->connection);

SSL_do_handshake [ n=0 / sslerr=1 / err=8 ]

# mkdir /tmp/core
# chmod 0755 /tmp/core
# chown www:www /tmp/core
# sysctl kern.sugid_coredump=1
# sysctl kern.corefile=/tmp/core/%N.core.%P
# gdb /usr/local/sbin/nginx /tmp/core/nginx.core.36451
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...
Core was generated by `nginx'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libcrypt.so.5...done.
Loaded symbols for /lib/libcrypt.so.5
Reading symbols from /usr/local/lib/libpcre.so.3...done.
Loaded symbols for /usr/local/lib/libpcre.so.3
Reading symbols from /lib/libz.so.6...done.
Loaded symbols for /lib/libz.so.6
Reading symbols from /usr/local/lib/libgd.so.5...done.
Loaded symbols for /usr/local/lib/libgd.so.5
Reading symbols from /lib/libc.so.7...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /lib/libthr.so.3...done.
Loaded symbols for /lib/libthr.so.3
Reading symbols from /usr/local/lib/libjpeg.so.11...done.
Loaded symbols for /usr/local/lib/libjpeg.so.11
Reading symbols from /lib/libm.so.5...done.
Loaded symbols for /lib/libm.so.5
Reading symbols from /usr/local/lib/libpng15.so.15...done.
Loaded symbols for /usr/local/lib/libpng15.so.15
Reading symbols from /usr/local/lib/libfreetype.so.6...done.
Loaded symbols for /usr/local/lib/libfreetype.so.6
Reading symbols from /usr/local/lib/libfontconfig.so.1...done.
Loaded symbols for /usr/local/lib/libfontconfig.so.1
Reading symbols from /usr/local/lib/libtiff.so.4...done.
Loaded symbols for /usr/local/lib/libtiff.so.4
Reading symbols from /usr/lib/libbz2.so.4...done.
Loaded symbols for /usr/lib/libbz2.so.4
Reading symbols from /usr/local/lib/libexpat.so.6...done.
Loaded symbols for /usr/local/lib/libexpat.so.6
Reading symbols from /usr/lib/liblzma.so.5...done.
Loaded symbols for /usr/lib/liblzma.so.5
Reading symbols from /usr/local/lib/libjbig.so.2...done.
Loaded symbols for /usr/local/lib/libjbig.so.2
Reading symbols from /libexec/ld-elf.so.1...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x0000000801453ba0 in strncmp () from /lib/libc.so.7
[New Thread 803406400 (LWP 100719/<unknown>)]
[New LWP 101618]
(gdb) set logging on
Copying output to gdb.txt.
(gdb) backtrace full
#0  0x0000000801453ba0 in strncmp () from /lib/libc.so.7
No symbol table info available.
#1  0x000000000051e9cf in ssl_parse_clienthello_tlsext ()
No symbol table info available.
#2  0x000000000050c6dd in ssl3_get_client_hello ()
No symbol table info available.
#3  0x000000000050b552 in ssl3_accept ()
No symbol table info available.
#4  0x000000000050501b in ssl23_accept ()
No symbol table info available.
#5  0x00000000004510bb in ngx_ssl_handshake (c=0x803620180) at src/event/ngx_event_openssl.c:1012
	n = 0
	sslerr = 1
	err = 8
#6  0x000000000046c503 in ngx_http_ssl_handshake (rev=0x8036578c0) at src/http/ngx_http_request.c:717
	p = (u_char *) 0x10036627f0 <Address 0x10036627f0 out of bounds>
	buf = "\026\000\000\000\000\000\000\000�����\177\000\000�����\177\000\000��D\000\000\000\000\0000����\177\000\000�zI\000\000\000\000\000h\224F\003\b\000\000\000��a\003\b\000\000\000\200\001b\003\b\000\000\000\030\210F\003\b\000\000\000�8g\003\b\000\000\000�xe\003\b\000\000\000�J\005\030+\000\000\000��\203"
	size = 1
	n = 1
	err = 0
	rc = 34416492928
	c = (ngx_connection_t *) 0x803620180
	hc = (ngx_http_connection_t *) 0x80377feb8
	sscf = (ngx_http_ssl_srv_conf_t *) 0x8034c7940
#7  0x000000000043c5f0 in ngx_event_process_posted (cycle=0x803468050, posted=0x83a8a8) at src/event/ngx_event_posted.c:40
	ev = (ngx_event_t *) 0x8036578c0
#8  0x000000000043ad1f in ngx_process_events_and_timers (cycle=0x803468050) at src/event/ngx_event.c:275
	flags = 3
	timer = 403
	delta = 1
#9  0x000000000044aaf1 in ngx_worker_process_cycle (cycle=0x803468050, data=0x1) at src/os/unix/ngx_process_cycle.c:822
	worker = 1
	i = 140737488344352
	c = (ngx_connection_t *) 0x7fffffffd630
#10 0x00000000004464ee in ngx_spawn_process (cycle=0x803468050, proc=0x44a900 <ngx_worker_process_cycle>, data=0x1, name=0x5b6966 "worker process", respawn=-3)
    at src/os/unix/ngx_process.c:198
	on = 1
	pid = 0
	s = 1
#11 0x00000000004482f9 in ngx_start_worker_processes (cycle=0x803468050, n=4, type=-3) at src/os/unix/ngx_process_cycle.c:368
	i = 1
	ch = {command = 1, pid = 36450, slot = 0, fd = 3}
#12 0x0000000000447af6 in ngx_master_process_cycle (cycle=0x803468050) at src/os/unix/ngx_process_cycle.c:140
	title = 0x803615548 "master process /usr/local/sbin/nginx"
	p = (u_char *) 0x80361556c ""
	size = 37
	i = 1
	n = 140737488345152
---Type <return> to continue, or q <return> to quit---
	sigio = 34414690408
	set = {__bits = {0, 0, 0, 0}}
	itv = {it_interval = {tv_sec = 0, tv_usec = 0}, it_value = {tv_sec = 6, tv_usec = 0}}
	live = 0
	delay = 0
	ls = (ngx_listening_t *) 0x0
	ccf = (ngx_core_conf_t *) 0x803469158
#13 0x0000000000408ee9 in main (argc=1, argv=0x7fffffffdae8) at src/core/nginx.c:407
	i = 54
	log = (ngx_log_t *) 0x836a10
	cycle = (ngx_cycle_t *) 0x803468050
	init_cycle = {conf_ctx = 0x0, pool = 0x803406800, log = 0x836a10, new_log = {log_level = 0, file = 0x0, connection = 0, handler = 0, data = 0x0, writer = 0, wdata = 0x0,
    action = 0x0, next = 0x0}, log_use_stderr = 0, files = 0x0, free_connections = 0x0, free_connection_n = 0, reusable_connections_queue = {prev = 0x0, next = 0x0}, listening = {
    elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, paths = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, open_files = {last = 0x0, part = {elts = 0x0,
      nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, shared_memory = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0},
  connection_n = 0, files_n = 0, connections = 0x0, read_events = 0x0, write_events = 0x0, old_cycle = 0x0, conf_file = {len = 31,
    data = 0x5b2e87 "/usr/local/etc/nginx/nginx.conf"}, conf_param = {len = 0, data = 0x0}, conf_prefix = {len = 21, data = 0x5b2e87 "/usr/local/etc/nginx/nginx.conf"}, prefix = {
    len = 21, data = 0x5b2e71 "/usr/local/etc/nginx/"}, lock_file = {len = 0, data = 0x0}, hostname = {len = 0, data = 0x0}}
	ccf = (ngx_core_conf_t *) 0x803469158

[Maxim Dounin]

From the backtrace it looks like the problem is in BoringSSL, and quick look into the code confirms this. From the ssl_scan_clienthello_tlsext() code (looks like it's inlined into ssl_parse_clienthello_tlsext() mentioned in the backtrace):

                        /* Decode each ServerName in the extension. */
                        while (CBS_len(&server_name_list) > 0)
                                {
                                uint8_t name_type;
                                CBS host_name;

                                /* Decode the NameType. */
                                if (!CBS_get_u8(&server_name_list, &name_type))
                                        {
                                        *out_alert = SSL_AD_DECODE_ERROR;
                                        return 0;
                                        }

                                /* Only host_name is supported. */
                                if (name_type != TLSEXT_NAMETYPE_host_name)
                                        continue;

                                if (!s->hit)
                                        {
                                        ...
                                        }
                                        }
                                else
                                        {
                                        s->servername_done = s->session->tlsext_hostname
                                                && strlen(s->session->tlsext_hostname) == CBS_len(&host_name)
                                                && strncmp(s->session->tlsext_hostname,
                                                        (char *)CBS_data(&host_name), CBS_len(&host_name)) == 0;
                                        }
                                }

Here host_name is used uninitialized in the else clause. If it happens to have len matching the length of the previously established session, segmentation fault will likely follow due to dereferencing uninitialized pointer. Feel free to report this to BoringSSL guys.