Opened 11 years ago
Closed 11 years ago
#489 closed defect (invalid)
Segfault in ngx_log_error_core under load
| Reported by: | Quanah Gibson-Mount | Owned by: | |
|---|---|---|---|
| Priority: | major | Milestone: | |
| Component: | nginx-core | Version: | 1.2.x |
| Keywords: | Cc: | ||
| uname -a: | Linux edge01 2.6.32-358.23.2.el6.x86_64 #1 SMP Wed Oct 16 19:01:37 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux | ||
| nginx -V: |
nginx version: nginx/1.2.0-zimbra
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) TLS SNI support enabled configure arguments: --prefix=/opt/zimbra/nginx-1.2.0-zimbra --with-cc-opt='-g -I/opt/zimbra/cyrus-sasl-2.1.26.2z/include -I/opt/zimbra/openssl-1.0.1e/include' --with-ld-opt='-L/opt/zimbra/cyrus-sasl-2.1.26.2z/lib -L/opt/zimbra/openssl-1.0.1e/lib' --with-debug --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-pcre --with-http_upstream_zmauth_module --with-http_zm_sso_module --with-mail --with-mail-sasl --with-mail_ssl_module --error-log-path=/opt/zimbra/log/nginx.log --http-log-path=/opt/zimbra/log/nginx.access.log --http-client-body-temp-path=/opt/zimbra/data/tmp/nginx/client --http-proxy-temp-path=/opt/zimbra/data/tmp/nginx/proxy --http-fastcgi-temp-path=/opt/zimbra/data/tmp/nginx/fastcgi --without-http_scgi_module --without-http_uwsgi_module --add-module=/home/build/p4/zimbra/IRONMAIDEN/ThirdParty/nginx/nviennot-nginx-tcp-keepalive --add-module=/home/build/p4/zimbra/IRONMAIDEN/ThirdParty/nginx/chunkin-nginx-module |
||
Description
When running nginx under load, we can routinely trigger a segfault. While the version is older (1.2.0) the underlying code in nginx looks unchanged in current releases as well.
Core was generated by `nginx: worker process '.
Program terminated with signal 11, Segmentation fault.
#0 0x0000000000407ce7 in ngx_log_error_core (level=8, log=0x2112050, err=0, fmt=0x4de4e7 "free: %p, unused: %uz") at src/core/ngx_log.c:98
98 src/core/ngx_log.c: No such file or directory.
in src/core/ngx_log.c
Missing separate debuginfos, use: debuginfo-install zimbra-proxy-8.0.4_GA_5737.RHEL6_64-20130524120036.x86_64
(gdb) bt
#0 0x0000000000407ce7 in ngx_log_error_core (level=8, log=0x2112050, err=0, fmt=0x4de4e7 "free: %p, unused: %uz") at src/core/ngx_log.c:98
#1 0x0000000000408b94 in ngx_destroy_pool (pool=0x20c9350) at src/core/ngx_palloc.c:76
#2 0x000000000042da36 in ngx_memcache_any_read_handler (rev=0x7effb47b8760) at src/core/ngx_memcache.c:1687
#3 0x000000000043640d in ngx_event_process_posted (cycle=0x2031230, posted=0x71ade8) at src/event/ngx_event_posted.c:40
#4 0x00000000004341b4 in ngx_process_events_and_timers (cycle=0x2031230) at src/event/ngx_event.c:274
#5 0x0000000000441666 in ngx_worker_process_cycle (cycle=0x2031230, data=0x0) at src/os/unix/ngx_process_cycle.c:871
#6 0x000000000043dbca in ngx_spawn_process (cycle=0x2031230, proc=0x4414aa <ngx_worker_process_cycle>, data=0x0, name=0x4e364c "worker process", respawn=-3) at src/os/unix/ngx_process.c:198
#7 0x00000000004404de in ngx_start_worker_processes (cycle=0x2031230, n=4, type=-3) at src/os/unix/ngx_process_cycle.c:430
#8 0x000000000043f832 in ngx_master_process_cycle (cycle=0x2031230) at src/os/unix/ngx_process_cycle.c:142
#9 0x0000000000405f66 in main (argc=3, argv=0x7fffa6608948) at src/core/nginx.c:410
(gdb) p log
$1 = (ngx_log_t *) 0x2112050
(gdb) p *log
$2 = {log_level = 3563600084739519333, file = 0x7340363533393032, connection = 8315180326215578228, handler = 0x706d6178652e3174, data = 0xa0d6d6f632e656c,
action = 0x676f682053534150 <Address 0x676f682053534150 out of bounds>}
The traceback clearly shows that the problem is related to ngx_memcache_any_read_handler(), which isn't nginx code, but a 3rd party modification.