id summary reporter owner description type status priority milestone component version resolution keywords cc uname nginx_version 1633 Zone configuration in upstream gives signal 11 on heavy loaded server https://stackoverflow.com/users/6797850/strangedata "I've been testing shared memory zone for an upstream configuration for a few days in a test environment without issues. When the same configuration was applied to our production servers, Nginx worker process started dying almost immediately, with signal 11. My upstream configuration looks like this: upstream backend_servers { zone backend_upstreams_zone 1m; sticky name=route path=/ hash=index expires=30m; server backend01-tst:28080; server backend02-tst:28080; server backend03-tst:28080; server backend04-tst:28080; server backend05-tst:28080; server backend06-tst:28080; server backend07-tst:28080; server backend08-tst:28080; server backend09-tst:28080; } I cloned the VM to another network, just to test the zone config in an exactly equal configuration, but without load the problem doesn't happen. I tried also playing with the zone size, up to 1024m, without success. Here are my env info: Linux frontend01-prd 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux nginx version: nginx/1.15.3 built by gcc 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3) built with OpenSSL 1.0.1f 6 Jan 2014 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie' --add-module=/opt/nginx-sources/nginx-upload-progress-module-master --add-module=/opt/nginx-sources/nginx-http-slice-master --add-module=/opt/nginx-sources/nginx-upload-module-2.255 --add-module=/opt/nginx-sources/nginx-goodies-nginx-sticky-module-ng-08a395c66e42 --add-module=/opt/nginx-sources/ngx_http_substitutions_filter_module-master --add-module=/opt/nginx-sources/ngx_dynamic_upstream-master GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type ""show copying"" and ""show warranty"" for details. This GDB was configured as ""x86_64-linux-gnu"". Type ""show configuration"" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type ""help"". Type ""apropos word"" to search for commands related to ""word""... Reading symbols from /usr/sbin/nginx...Reading symbols from /usr/lib/debug/.build-id/00/6543010a42d5e472fb38a40895ce2f44b3e279.debug...done. done. BFD: Warning: /dados/coredumps/core-2018-09-11 is truncated: expected core file size >= 5505687552, found: 1073741824. [New LWP 2364] [New LWP 2365] [New LWP 2366] Failed to read a valid object file image from memory. Core was generated by `nginx: worker process '. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007feeebef6a99 in ngx_event_connect_peer (pc=0x7feeece3e9b8, pc@entry=) at src/event/ngx_event_connect.c:41 41 src/event/ngx_event_connect.c: No such file or directory. (gdb) bt full #0 0x00007feeebef6a99 in ngx_event_connect_peer (pc=0x7feeece3e9b8, pc@entry=) at src/event/ngx_event_connect.c:41 rc = type = 1 port = err = level = s = rev = wev = c = Cannot access memory at address 0x7fff742d4338 Unfortunately I cannot disable the modules to test in our production environment, and I cannot reproduce the issue without the heavy load we get in production. Even a 1000 threads JMeter isn't enough to trigger it." defect closed major nginx-core 1.15.x invalid Linux frontend01-prd 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux "nginx version: nginx/1.15.3 built by gcc 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3) built with OpenSSL 1.0.1f 6 Jan 2014 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie' --add-module=/opt/nginx-sources/nginx-upload-progress-module-master --add-module=/opt/nginx-sources/nginx-http-slice-master --add-module=/opt/nginx-sources/nginx-upload-module-2.255 --add-module=/opt/nginx-sources/nginx-goodies-nginx-sticky-module-ng-08a395c66e42 --add-module=/opt/nginx-sources/ngx_http_substitutions_filter_module-master --add-module=/opt/nginx-sources/ngx_dynamic_upstream-master"