Opened 8 years ago
Closed 8 years ago
#1097 closed defect (fixed)
Nginx worker process exited on signal 11
Reported by: | Owned by: | ||
---|---|---|---|
Priority: | major | Milestone: | |
Component: | nginx-core | Version: | 1.10.x |
Keywords: | Cc: | ||
uname -a: | FreeBSD fe1.sports.ru 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 | ||
nginx -V: |
nginx version: nginx/1.10.1
built with OpenSSL 1.0.2h 3 May 2016 TLS SNI support enabled configure arguments: --prefix=/usr/local/etc/nginx --with-cc-opt='-I /usr/local/include' --with-ld-opt='-L /usr/local/lib' --conf-path=/usr/local/etc/nginx/nginx.conf --sbin-path=/usr/local/sbin/nginx --pid-path=/var/run/nginx.pid --error-log-path=/var/log/nginx/error.log --user=www --group=www --with-file-aio --http-client-body-temp-path=/var/tmp/nginx/client_body_temp --http-fastcgi-temp-path=/var/tmp/nginx/fastcgi_temp --http-proxy-temp-path=/var/tmp/nginx/proxy_temp --http-scgi-temp-path=/var/tmp/nginx/scgi_temp --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi_temp --http-log-path=/var/log/nginx/access.log --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_realip_module --with-http_slice_module --with-http_stub_status_module --with-http_sub_module --with-pcre --with-http_v2_module --with-stream --with-stream_ssl_module --with-http_ssl_module |
Description (last modified by )
I have 2 nginx frontends worked in load balancing mode.
Today, around the same time on both frontends I got error: worker process exited on signal 11
Backtrace for both core dumps are similar. Backtrace are below.
#0 0x0000000000000000 in ?? () No symbol table info available. #1 0x0000000000457a37 in ngx_kqueue_process_events (cycle=0x8024b3050, timer=23, flags=1) at src/event/modules/ngx_kqueue_module.c:669 events = 512 n = 85 i = 501 instance = 0 level = 140737488349120 err = 0 ev = (ngx_event_t *) 0x806bd7478 queue = (ngx_queue_t *) 0x4ab0b1 ts = {tv_sec = 0, tv_nsec = 23000000} tp = (struct timespec *) 0x7fffffffe758 #2 0x00000000004482fe in ngx_process_events_and_timers (cycle=0x8024b3050) at src/event/ngx_event.c:242 flags = 1 timer = 23 delta = 1475766275958 #3 0x0000000000455aa3 in ngx_worker_process_cycle (cycle=0x8024b3050, data=0x3) at src/os/unix/ngx_process_cycle.c:753 worker = 3 #4 0x0000000000451d30 in ngx_spawn_process (cycle=0x8024b3050, proc=0x4559d0 <ngx_worker_process_cycle>, data=0x3, name=0x523cd6 "worker process", respawn=-4) at src/os/unix/ngx_process.c:198 on = 1 pid = 0 s = 20 #5 0x0000000000453a29 in ngx_start_worker_processes (cycle=0x8024b3050, n=16, type=-4) at src/os/unix/ngx_process_cycle.c:358 i = 3 ch = {command = 1, pid = 40518, slot = 19, fd = 8} #6 0x00000000004537be in ngx_master_process_cycle (cycle=0x8024b3050) at src/os/unix/ngx_process_cycle.c:243 title = 0x8097e7b9c "\b" p = (u_char *) 0x8097e7bc0 "\020" size = 37 i = 1 n = 140737488349776 sigio = 0 set = {__bits = {0, 0, 0, 0}} itv = {it_interval = {tv_sec = 0, tv_usec = 0}, it_value = {tv_sec = 6, tv_usec = 0}} live = 1 delay = 0 ls = (ngx_listening_t *) 0x0 ccf = (ngx_core_conf_t *) 0x8024b4b58 #7 0x00000000004129f8 in main (argc=1, argv=0x7fffffffed60) at src/core/nginx.c:367 b = (ngx_buf_t *) 0x800745c00 log = (ngx_log_t *) 0x75b648 i = 34386931443 cycle = (ngx_cycle_t *) 0x802483050 init_cycle = {conf_ctx = 0x0, pool = 0x802406800, log = 0x75b648, new_log = {log_level = 0, file = 0x0, connection = 0, disk_full_time = 0, handler = 0, data = 0x0, writer = 0, wdata = 0x0, action = 0x0, next = 0x0}, log_use_stderr = 0, files = 0x0, free_connections = 0x0, free_connection_n = 0, modules = 0x0, modules_n = 0, modules_used = 0, reusable_connections_queue = {prev = 0x0, next = 0x0}, listening = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, paths = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, config_dump = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, open_files = {last = 0x0, part = { elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, shared_memory = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, connection_n = 0, files_n = 0, connections = 0x0, read_events = 0x0, write_events = 0x0, old_cycle = 0x0, conf_file = {len = 31, data = 0x520358 "/usr/local/etc/nginx/nginx.conf"}, conf_param = {len = 0, data = 0x0}, conf_prefix = {len = 21, data = 0x520358 "/usr/local/etc/nginx/nginx.conf"}, prefix = {len = 21, data = 0x520342 "/usr/local/etc/nginx/"}, lock_file = {len = 0, data = 0x0}, hostname = {len = 0, data = 0x0}} cd = (ngx_conf_dump_t *) 0x80076a000 ccf = (ngx_core_conf_t *) 0x802484d08
Change History (4)
comment:1 by , 8 years ago
Description: | modified (diff) |
---|
comment:2 by , 8 years ago
comment:3 by , 8 years ago
Replying to adushein.sports.ru@…:
I have 2 nginx frontends worked in load balancing mode.
Today, around the same time on both frontends I got error: worker process exited on signal 11
Backtrace for both core dumps are similar. Backtrace are below.
#0 0x0000000000000000 in ?? () No symbol table info available. #1 0x0000000000457a37 in ngx_kqueue_process_events (cycle=0x8024b3050, timer=23, flags=1) at src/event/modules/ngx_kqueue_module.c:669 events = 512 n = 85 i = 501 instance = 0 level = 140737488349120 err = 0 ev = (ngx_event_t *) 0x806bd7478 queue = (ngx_queue_t *) 0x4ab0b1 ts = {tv_sec = 0, tv_nsec = 23000000} tp = (struct timespec *) 0x7fffffffe758 #2 0x00000000004482fe in ngx_process_events_and_timers (cycle=0x8024b3050) at src/event/ngx_event.c:242 flags = 1 timer = 23 delta = 1475766275958 #3 0x0000000000455aa3 in ngx_worker_process_cycle (cycle=0x8024b3050, data=0x3) at src/os/unix/ngx_process_cycle.c:753 worker = 3 #4 0x0000000000451d30 in ngx_spawn_process (cycle=0x8024b3050, proc=0x4559d0 <ngx_worker_process_cycle>, data=0x3, name=0x523cd6 "worker process", respawn=-4) at src/os/unix/ngx_process.c:198 on = 1 pid = 0 s = 20 #5 0x0000000000453a29 in ngx_start_worker_processes (cycle=0x8024b3050, n=16, type=-4) at src/os/unix/ngx_process_cycle.c:358 i = 3 ch = {command = 1, pid = 40518, slot = 19, fd = 8} #6 0x00000000004537be in ngx_master_process_cycle (cycle=0x8024b3050) at src/os/unix/ngx_process_cycle.c:243 title = 0x8097e7b9c "\b" p = (u_char *) 0x8097e7bc0 "\020" size = 37 i = 1 n = 140737488349776 sigio = 0 set = {__bits = {0, 0, 0, 0}} itv = {it_interval = {tv_sec = 0, tv_usec = 0}, it_value = {tv_sec = 6, tv_usec = 0}} live = 1 delay = 0 ls = (ngx_listening_t *) 0x0 ccf = (ngx_core_conf_t *) 0x8024b4b58 #7 0x00000000004129f8 in main (argc=1, argv=0x7fffffffed60) at src/core/nginx.c:367 b = (ngx_buf_t *) 0x800745c00 log = (ngx_log_t *) 0x75b648 i = 34386931443 cycle = (ngx_cycle_t *) 0x802483050 init_cycle = {conf_ctx = 0x0, pool = 0x802406800, log = 0x75b648, new_log = {log_level = 0, file = 0x0, connection = 0, disk_full_time = 0, handler = 0, data = 0x0, writer = 0, wdata = 0x0, action = 0x0, next = 0x0}, log_use_stderr = 0, files = 0x0, free_connections = 0x0, free_connection_n = 0, modules = 0x0, modules_n = 0, modules_used = 0, reusable_connections_queue = {prev = 0x0, next = 0x0}, listening = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, paths = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, config_dump = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, open_files = {last = 0x0, part = { elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, shared_memory = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, connection_n = 0, files_n = 0, connections = 0x0, read_events = 0x0, write_events = 0x0, old_cycle = 0x0, conf_file = {len = 31, data = 0x520358 "/usr/local/etc/nginx/nginx.conf"}, conf_param = {len = 0, data = 0x0}, conf_prefix = {len = 21, data = 0x520358 "/usr/local/etc/nginx/nginx.conf"}, prefix = {len = 21, data = 0x520342 "/usr/local/etc/nginx/"}, lock_file = {len = 0, data = 0x0}, hostname = {len = 0, data = 0x0}} cd = (ngx_conf_dump_t *) 0x80076a000 ccf = (ngx_core_conf_t *) 0x802484d08
comment:4 by , 8 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
Since no requested details are provided it is not possible to tell what exactly goes wrong here, though I suspect it is most likely the aio + X-Accel-Redirect problem as mentioned in 5850fed24639. Closing this as "fixed" accordingly.
Please provide output of the following gdb commands:
Please also provide full configuration used (can be easily obtained with
nginx -T
).