Opened 9 years ago

Closed 8 years ago

#981 closed defect (fixed)

Segmentation fault on chunked WebDAV upload

Reported by: Oleksandr Typlyns'kyi Owned by:
Priority: minor Milestone:
Component: nginx-core Version: 1.10.x
Keywords: Cc: vbart@…
uname -a: FreeBSD 10.3-RELEASE-p2 amd64
nginx -V: nginx version: nginx/1.10.0
configure arguments: --add-module=/root/src/ngx_http_ip_tos_filter_module --prefix=/usr/local/etc/nginx --with-cc-opt='-I /usr/local/include' --with-ld-opt='-L /usr/local/lib' --conf-path=/usr/local/etc/nginx/nginx.conf --sbin-path=/usr/local/sbin/nginx --pid-path=/var/run/nginx.pid --error-log-path=/var/log/nginx-error.log --user=www --group=www --http-client-body-temp-path=/var/tmp/nginx/client_body_temp --http-fastcgi-temp-path=/var/tmp/nginx/fastcgi_temp --http-proxy-temp-path=/var/tmp/nginx/proxy_temp --http-scgi-temp-path=/var/tmp/nginx/scgi_temp --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi_temp --http-log-path=/var/log/nginx-access.log --with-http_dav_module --add-module=/usr/ports/www/nginx-devel/work/nginx-dav-ext-module-0.0.3 --with-http_flv_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_realip_module --add-module=/usr/ports/www/nginx-devel/work/ngx_http_substitutions_filter_module-0.6.4 --with-http_secure_link_module --with-http_stub_status_module --with-http_sub_module --with-pcre

Description

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...
Core was generated by `nginx'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libcrypt.so.5...done.
Loaded symbols for /lib/libcrypt.so.5
Reading symbols from /usr/local/lib/libexpat.so.1...done.
Loaded symbols for /usr/local/lib/libexpat.so.1
Reading symbols from /usr/local/lib/libpcre.so.1...done.
Loaded symbols for /usr/local/lib/libpcre.so.1
Reading symbols from /lib/libmd.so.6...done.
Loaded symbols for /lib/libmd.so.6
Reading symbols from /lib/libz.so.6...done.
Loaded symbols for /lib/libz.so.6
Reading symbols from /lib/libthr.so.3...done.
Loaded symbols for /lib/libthr.so.3
Reading symbols from /usr/local/lib/perl5/5.20/mach/CORE/libperl.so.5.20...done.
Loaded symbols for /usr/local/lib/perl5/5.20/mach/CORE/libperl.so.5.20
Reading symbols from /lib/libm.so.5...done.
Loaded symbols for /lib/libm.so.5
Reading symbols from /lib/libutil.so.9...done.
Loaded symbols for /lib/libutil.so.9
Reading symbols from /lib/libc.so.7...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /usr/local/lib/perl5/site_perl/mach/5.20/auto/nginx/nginx.so...done.
Loaded symbols for /usr/local/lib/perl5/site_perl/mach/5.20/auto/nginx/nginx.so
Reading symbols from /libexec/ld-elf.so.1...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x00000000004499c2 in ngx_chain_to_iovec (vec=0x7fffffffe538, cl=0x802841a20) at src/os/unix/ngx_files.c:362
362                 iov->iov_len += size;
[New Thread 802806400 (LWP 100458/<unknown>)]
(gdb) bt full
#0  0x00000000004499c2 in ngx_chain_to_iovec (vec=0x7fffffffe538, cl=0x802841a20) at src/os/unix/ngx_files.c:362
        total = 0
        size = 0
        prev = (u_char *) 0x0
        n = 0
        iov = (struct iovec *) 0x0
#1  0x0000000000449880 in ngx_write_chain_to_file (file=0x802841aa0, cl=0x802841a20, offset=3143360, pool=0x802859000) at src/os/unix/ngx_files.c:315
        total = 1047238
        n = 1047238
        vec = {iovs = 0x7fffffffe130, count = 64, size = 1047238, nalloc = 64}
        iovs = {{iov_base = 0x803506000, iov_len = 15866}, {iov_base = 0x803509e02, iov_len = 16372}, {iov_base = 0x80350ddfe, iov_len = 16372}, {iov_base = 0x803511dfa, iov_len = 16372}, {
    iov_base = 0x803515df6, iov_len = 16372}, {iov_base = 0x803519df2, iov_len = 16372}, {iov_base = 0x80351ddee, iov_len = 16372}, {iov_base = 0x803521dea, iov_len = 16372}, {iov_base = 0x803525de6,
    iov_len = 16372}, {iov_base = 0x803529de2, iov_len = 16372}, {iov_base = 0x80352ddde, iov_len = 16372}, {iov_base = 0x803531dda, iov_len = 16372}, {iov_base = 0x803535dd6, iov_len = 16372}, {
    iov_base = 0x803539dd2, iov_len = 16372}, {iov_base = 0x80353ddce, iov_len = 16372}, {iov_base = 0x803541dca, iov_len = 16372}, {iov_base = 0x803545dc6, iov_len = 16372}, {iov_base = 0x803549dc2,
    iov_len = 16372}, {iov_base = 0x80354ddbe, iov_len = 16372}, {iov_base = 0x803551dba, iov_len = 16372}, {iov_base = 0x803555db6, iov_len = 16372}, {iov_base = 0x803559db2, iov_len = 16372}, {
    iov_base = 0x80355ddae, iov_len = 16372}, {iov_base = 0x803561daa, iov_len = 16372}, {iov_base = 0x803565da6, iov_len = 16372}, {iov_base = 0x803569da2, iov_len = 16372}, {iov_base = 0x80356dd9e,
    iov_len = 16372}, {iov_base = 0x803571d9a, iov_len = 16372}, {iov_base = 0x803575d96, iov_len = 16372}, {iov_base = 0x803579d92, iov_len = 16372}, {iov_base = 0x80357dd8e, iov_len = 16372}, {
    iov_base = 0x803581d8a, iov_len = 16372}, {iov_base = 0x803585d86, iov_len = 16372}, {iov_base = 0x803589d82, iov_len = 16372}, {iov_base = 0x80358dd7e, iov_len = 16372}, {iov_base = 0x803591d7a,
    iov_len = 16372}, {iov_base = 0x803595d76, iov_len = 16372}, {iov_base = 0x803599d72, iov_len = 16372}, {iov_base = 0x80359dd6e, iov_len = 16372}, {iov_base = 0x8035a1d6a, iov_len = 16372}, {
    iov_base = 0x8035a5d66, iov_len = 16372}, {iov_base = 0x8035a9d62, iov_len = 16372}, {iov_base = 0x8035add5e, iov_len = 16372}, {iov_base = 0x8035b1d5a, iov_len = 16372}, {iov_base = 0x8035b5d56,
    iov_len = 16372}, {iov_base = 0x8035b9d52, iov_len = 16372}, {iov_base = 0x8035bdd4e, iov_len = 16372}, {iov_base = 0x8035c1d4a, iov_len = 16372}, {iov_base = 0x8035c5d46, iov_len = 16372}, {
    iov_base = 0x8035c9d42, iov_len = 16372}, {iov_base = 0x8035cdd3e, iov_len = 16372}, {iov_base = 0x8035d1d3a, iov_len = 16372}, {iov_base = 0x8035d5d36, iov_len = 16372}, {iov_base = 0x8035d9d32,
    iov_len = 16372}, {iov_base = 0x8035ddd2e, iov_len = 16372}, {iov_base = 0x8035e1d2a, iov_len = 16372}, {iov_base = 0x8035e5d26, iov_len = 16372}, {iov_base = 0x8035e9d22, iov_len = 16372}, {
    iov_base = 0x8035edd1e, iov_len = 16372}, {iov_base = 0x8035f1d1a, iov_len = 16372}, {iov_base = 0x8035f5d16, iov_len = 16372}, {iov_base = 0x8035f9d12, iov_len = 16372}, {iov_base = 0x8035fdd0e,
    iov_len = 16372}, {iov_base = 0x803601d0a, iov_len = 16308}}
#2  0x00000000004204ab in ngx_write_chain_to_temp_file (tf=0x802841aa0, chain=0x802859e48) at src/core/ngx_file.c:136
        rc = 0
#3  0x0000000000478370 in ngx_http_write_request_body (r=0x802859050) at src/http/ngx_http_request_body.c:483
        n = 34402049224
        cl = (ngx_chain_t *) 0x0
        ln = (ngx_chain_t *) 0x802841a20
        tf = (ngx_temp_file_t *) 0x802841a28
        rb = (ngx_http_request_body_t *) 0x802859cc0
        clcf = (ngx_http_core_loc_conf_t *) 0x7fffffffe610
#4  0x000000000047800c in ngx_http_request_body_save_filter (r=0x802859050, in=0x8028419c0) at src/http/ngx_http_request_body.c:1137
        b = (ngx_buf_t *) 0x8028419c0
        cl = (ngx_chain_t *) 0x803605cc4
        rb = (ngx_http_request_body_t *) 0x802859cc0
#5  0x000000000047894e in ngx_http_request_body_chunked_filter (r=0x802859050, in=0x7fffffffe730) at src/http/ngx_http_request_body.c:1068
        size = 140737488348928
        rc = -4
        b = (ngx_buf_t *) 0x8028419d0
        cl = (ngx_chain_t *) 0x0
        out = (ngx_chain_t *) 0x8028419c0
        tl = (ngx_chain_t *) 0x8028419c0
        ll = (ngx_chain_t **) 0x8028419c8
        rb = (ngx_http_request_body_t *) 0x802859cc0
        clcf = (ngx_http_core_loc_conf_t *) 0x802898020
#6  0x0000000000476c9f in ngx_http_request_body_filter (r=0x802859050, in=0x7fffffffe730) at src/http/ngx_http_request_body.c:846
No locals.
#7  0x00000000004770b0 in ngx_http_do_read_client_request_body (r=0x802859050) at src/http/ngx_http_request_body.c:364
        rest = 1
        size = 1
        n = 1
        rc = 0
        out = {buf = 0x802859d18, next = 0x0}
        c = (ngx_connection_t *) 0x802c0ab18
        rb = (ngx_http_request_body_t *) 0x802859cc0
        clcf = (ngx_http_core_loc_conf_t *) 0x802898020
#8  0x0000000000476d3f in ngx_http_read_client_request_body_handler (r=0x802859050) at src/http/ngx_http_request_body.c:255
        rc = 34401935440
#9  0x0000000000469640 in ngx_http_request_handler (ev=0x802ad6428) at src/http/ngx_http_request.c:2194
        c = (ngx_connection_t *) 0x802c0ab18
        r = (ngx_http_request_t *) 0x802859050
#10 0x00000000004449be in ngx_event_process_posted (cycle=0x802852050, posted=0x73bce0) at src/event/ngx_event_posted.c:33
        q = (ngx_queue_t *) 0x802ad6480
        ev = (ngx_event_t *) 0x802ad6428
#11 0x0000000000443642 in ngx_process_events_and_timers (cycle=0x802852050) at src/event/ngx_event.c:259
        flags = 3
        timer = 9613
        delta = 0
#12 0x0000000000450c93 in ngx_worker_process_cycle (cycle=0x802852050, data=0x1) at src/os/unix/ngx_process_cycle.c:753
        worker = 1
#13 0x000000000044cf20 in ngx_spawn_process (cycle=0x802852050, proc=0x450bc0 <ngx_worker_process_cycle>, data=0x1, name=0x4fe60e "worker process", respawn=1) at src/os/unix/ngx_process.c:198
        on = 1
        pid = 0
        s = 1
#14 0x000000000044f1da in ngx_reap_children (cycle=0x802852050) at src/os/unix/ngx_process_cycle.c:621
        i = 1
        n = 14
        live = 1
        ch = {command = 2, pid = 8926, slot = 1, fd = -1}
        ccf = (ngx_core_conf_t *) 0x72a96c
#15 0x000000000044e6b1 in ngx_master_process_cycle (cycle=0x802852050) at src/os/unix/ngx_process_cycle.c:174
        title = 0x802abe01c "master process /usr/local/sbin/nginx"
        p = (u_char *) 0x802abe040 ""
        size = 37
        i = 1
        n = 34392181856
        sigio = 0
        set = {__bits = {0, 0, 0, 0}}
        itv = {it_interval = {tv_sec = 2, tv_usec = 4}, it_value = {tv_sec = 5, tv_usec = 34402018096}}
        live = 1
        delay = 0
        ls = (ngx_listening_t *) 0x0
        ccf = (ngx_core_conf_t *) 0x802853d80
#16 0x000000000040df68 in main (argc=1, argv=0x7fffffffed90) at src/core/nginx.c:367
        b = (ngx_buf_t *) 0x80071bc00
        log = (ngx_log_t *) 0x728e28
        i = 34392551155
        cycle = (ngx_cycle_t *) 0x802852050
        init_cycle = {conf_ctx = 0x0, pool = 0x802806800, log = 0x728e28, new_log = {log_level = 0, file = 0x0, connection = 0, disk_full_time = 0, handler = 0, data = 0x0, writer = 0, wdata = 0x0,
    action = 0x0, next = 0x0}, log_use_stderr = 0, files = 0x0, free_connections = 0x0, free_connection_n = 0, modules = 0x0, modules_n = 0, modules_used = 0, reusable_connections_queue = {prev = 0x0,
    next = 0x0}, listening = {elts = 0x80284f000, nelts = 11, size = 256, nalloc = 20, pool = 0x802806800}, paths = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, config_dump = {elts = 0x0,
    nelts = 0, size = 0, nalloc = 0, pool = 0x0}, open_files = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, shared_memory = {last = 0x0, part = {elts = 0x0,
      nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, connection_n = 0, files_n = 0, connections = 0x0, read_events = 0x0, write_events = 0x0, old_cycle = 0x0, conf_file = {len = 31,
    data = 0x4fac88 "/usr/local/etc/nginx/nginx.conf"}, conf_param = {len = 0, data = 0x0}, conf_prefix = {len = 21, data = 0x4fac88 "/usr/local/etc/nginx/nginx.conf"}, prefix = {len = 21,
    data = 0x4fac72 "/usr/local/etc/nginx/"}, lock_file = {len = 0, data = 0x0}, hostname = {len = 0, data = 0x0}}
        cd = (ngx_conf_dump_t *) 0x80073c800
        ccf = (ngx_core_conf_t *) 0x802853d80
(gdb) up 8
#8  0x0000000000476d3f in ngx_http_read_client_request_body_handler (r=0x802859050) at src/http/ngx_http_request_body.c:255
255         rc = ngx_http_do_read_client_request_body(r);
(gdb) set $r=((ngx_http_request_t *) r)
(gdb) print $r->uri
$1 = {len = 56, data = 0x802812804 "/hls-tv/fashiontvhd/fashiontvhd20160516T0541494_00381.ts HTTP/1.1\r\nHost"}
(gdb) print $r->request_line
$2 = {len = 69, data = 0x802812800 "PUT /hls-tv/fashiontvhd/fashiontvhd20160516T0541494_00381.ts HTTP/1.1\r\nHost"}
(gdb) print *$r->request_body
$3 = {temp_file = 0x802841aa0, bufs = 0x802859e48, buf = 0x802859d18, rest = 0, free = 0x802841a30, busy = 0x802859de8, chunked = 0x802859d00, post_handler = 0x4b22d0 <ngx_http_dav_put_handler>}

С 1.9.10 не падает, в tcpdump в заголовках ничего плохого не показывает:

PUT /hls-tv/fashiontvhd/fashiontvhd20160516T0645414_01339.ts HTTP/1.1
Host: 172.30.40.28
Accept: */*
Transfer-Encoding: chunked
User-Agent: Elemental 2.0.1.1421
Connection: Keep-Alive
Overwrite: T
Content-Type: video/MP2T
Expect: 100-continue

Change History (5)

comment:1 by Maxim Dounin, 8 years ago

Какой-то такой патч должен помочь:

# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1463412458 -10800
#      Mon May 16 18:27:38 2016 +0300
# Node ID e27d3d89026f164adf1b28a1918210c3ba4847fa
# Parent  db699978a33fe19d786e95e00143eb2b6a1662c0
Core: skip special buffers on writing (ticket #981).

diff --git a/src/os/unix/ngx_files.c b/src/os/unix/ngx_files.c
--- a/src/os/unix/ngx_files.c
+++ b/src/os/unix/ngx_files.c
@@ -356,6 +356,11 @@ ngx_chain_to_iovec(ngx_iovec_t *vec, ngx
     n = 0;
 
     for ( /* void */ ; cl; cl = cl->next) {
+
+        if (ngx_buf_special(cl->buf)) {
+            continue;
+        }
+
         size = cl->buf->last - cl->buf->pos;
 
         if (prev == cl->buf->pos) {

comment:2 by Oleksandr Typlyns'kyi, 8 years ago

За 40 минут работы больше не падал.
Файлы целые - md5 с загруженными на 1.9.10 совпадает.
Спасибо.

comment:3 by Maxim Dounin <mdounin@…>, 8 years ago

In 6574:7eb19447b2de/nginx:

Core: skip special buffers on writing (ticket #981).

A special last buffer with cl->buf->pos set to NULL can be present in
a chain when writing request body if chunked encoding was used. This
resulted in a NULL pointer dereference if it happened to be the only
buffer left after a do...while loop iteration in ngx_write_chain_to_file().

The problem originally appeared in nginx 1.3.9 with chunked encoding
support. Additionally, rev. 3832b608dc8d (nginx 1.9.13) changed the
minimum number of buffers to trigger this from IOV_MAX (typically 1024)
to NGX_IOVS_PREALLOCATE (typically 64).

Fix is to skip such buffers in ngx_chain_to_iovec(), much like it is
done in other places.

comment:4 by Maxim Dounin <mdounin@…>, 8 years ago

In 6579:0ac0575e955d/nginx:

Core: skip special buffers on writing (ticket #981).

A special last buffer with cl->buf->pos set to NULL can be present in
a chain when writing request body if chunked encoding was used. This
resulted in a NULL pointer dereference if it happened to be the only
buffer left after a do...while loop iteration in ngx_write_chain_to_file().

The problem originally appeared in nginx 1.3.9 with chunked encoding
support. Additionally, rev. 3832b608dc8d (nginx 1.9.13) changed the
minimum number of buffers to trigger this from IOV_MAX (typically 1024)
to NGX_IOVS_PREALLOCATE (typically 64).

Fix is to skip such buffers in ngx_chain_to_iovec(), much like it is
done in other places.

comment:5 by Maxim Dounin, 8 years ago

Resolution: fixed
Status: newclosed

Fix comitted.

Note: See TracTickets for help on using tickets.