Opened 8 years ago

Closed 8 years ago

#975 closed defect (invalid)

nginx worker die with trap divide error using upstream_conf with ip_hash

Reported by: guzman.braso@… Owned by:
Priority: minor Milestone:
Component: other Version: 1.5.x
Keywords: Cc:
uname -a: Linux qtsnginx02 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3 x86_64 GNU/Linux
nginx -V:
nginx version: nginx/1.5.12
built by gcc 4.7.2 (Debian 4.7.2-5)
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_f4f_module --with-http_hls_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_session_log_module --with-syslog --with-http_stub_status_module --with-http_auth_request_module --with-mail --with-mail_ssl_module --with-file-aio --with-http_spdy_module --with-ipv6 --with-http_perl_module --with-http_image_filter_module --with-http_geoip_module --with-http_xslt_module --add-module=debian/extra/ngx_devel_kit-0.2.19 --add-module=debian/extra/lua-nginx-module-0.9.4 --add-module=debian/extra/headers-more-nginx-module-0.25 --add-module=debian/extra/set-misc-nginx-module-0.24 --add-module=debian/extra/passenger-4.0.41/ext/nginx --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-z,relro -Wl,--as-needed'

Description

dpkg -l|grep nginx
ii nginx-plus-extras 1.5.12-1~wheezy amd64 NGINX Plus, provided by Nginx, Inc.
ii nginx-plus-extras-debug 1.5.12-1~wheezy amd64 NGINX Plus, provided by Nginx, Inc. (debug build)

How to reproduce

  • 1. Configure upstream with ip_hash and upstream_conf:

upstream N {

ip_hash ;
keepalive 100 ;
zone N 64k ;
server 1.1.1.1;
server 2.2.2.2;

}

  • 2. Remove all nodes without marking down

upstream_conf?upstream=N&id=0&remove=1
upstream_conf?upstream=N&id=1&remove=1

  • 3. Try to load an url and worker dies.

Dmesg logs:
[35010800.450094] nginx[9426] trap divide error ip:4d225b sp:7fffc98e8c40 error:0 in nginx[400000+230000]

Error logs:
2016/05/06 21:22:10 [alert] 9769#0: worker process 9770 exited on signal 8
2016/05/06 21:22:10 [alert] 9769#0: shared memory zone "N" was locked by 9770

I don't know if it happens with latest version, I do not have right now the time or hardware for testing on latest plus.

Change History (3)

comment:1 by guzman.braso@…, 8 years ago

If ip_hash is not used I can't reproduce the bug.

If instead of remove option of upstream_conf I use the down option to take backends offline bug does not reproduce.

comment:2 by grunt, 8 years ago

Thank you for your report.
I tried to reproduce the issue that you are facing on my test environment and I couldn't reproduce it. I have "no live upstreams" record in the error.log, which is expected behaviour of NGINX+ in this case. I used R7 and R9 versions in my tests.

As I can see from the report, you are using an old version of NGINX+ (R3) which is unsupported now. I strongly recommend you upgrading your NGINX+ installation to the latest stable release (which is R9 at the moment).

Furthermore, you are using an NGINX+ so I suggest you using an email address for NGINX+ customers for such questions. We are always glad to help.
Konstantin Romanenko, NGINX+ support engineer.

comment:3 by Maxim Dounin, 8 years ago

Resolution: invalid
Status: newclosed

This is a bug tracker for opensource nginx, so I'm closing this ticket. See Konstantin's answer for some related details, and please consider using nginx-plus support channels for further questions about nginx-plus.

Note: See TracTickets for help on using tickets.