Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#1100 closed defect (invalid)

Socket leakage after ngx_http_read_client_request_body

Reported by: Dmitry Utkin Owned by:
Priority: minor Milestone:
Component: other Version: 1.10.x
Keywords: Cc:
uname -a: Linux dem-B14 3.13.0-86-generic #130-Ubuntu SMP Mon Apr 18 18:27:15 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.10.1
built with OpenSSL 1.0.1f 6 Jan 2014
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-stream --with-stream_ssl_module --with-mail --with-mail_ssl_module --with-threads --add-module=/build/nginx-abUnII/nginx-1.10.1/debian/modules/nginx-auth-pam --add-module=/build/nginx-abUnII/nginx-1.10.1/debian/modules/nginx-dav-ext-module --add-module=/build/nginx-abUnII/nginx-1.10.1/debian/modules/nginx-echo --add-module=/build/nginx-abUnII/nginx-1.10.1/debian/modules/nginx-upstream-fair --add-module=/build/nginx-abUnII/nginx-1.10.1/debian/modules/ngx_http_substitutions_filter_module

Description

When handling POST request from module and ngx_http_read_client_request_body is called, there are one socket per handled request in CLOSE_WAIT state appeared.

How to reproduce:

0) install from ppa:

sudo add-apt-repository ppa:nginx/stable
# to enable source package
sudo sed -i 's/# //' /etc/apt/sources.list.d/nginx*-stable*.list
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install nginx
# get source code and configure it to build the module
mkdir ~/nginx-build && cd ~/nginx-build
apt-get source nginx
cd nginx-*
nginx -V 2>&1 | tr ' '  '\n' | grep -- --with | egrep -v -- '--with-(cc|ld)' | xargs ./configure

1) Unpack module attached, build it with GNU Make.

2) Edit /etc/nginx/nginx.conf, add line before config adjusting path:

load_module /path/to/mod_test_nginx.so;

3) Edit /etc/nginx/sites-enabled/default, add these lines inside server section:

location ~ /test_wait {
    test_wait;
}

4) restart Nginx

5) check sockets:

$ sudo lsof -i tcp -a -c nginx
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
COMMAND  PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
nginx   4930     root    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4931 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4932 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4934 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4935 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)

It's OK.

6) run CURL to the module and check sockets again:

$ curl -X POST -d 'test body' 'http://localhost/test_wait'
This is response

$ sudo lsof -i tcp -a -c nginx
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
COMMAND  PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
nginx   4930     root    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4931 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4931 www-data   12u  IPv4  92738      0t0  TCP localhost:http->localhost:46214 (CLOSE_WAIT)
nginx   4932 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4934 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4935 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)

The more requests you do, more sockets in CLOSE_WAIT state

After 8 more requests:

$ sudo lsof -i tcp -a -c nginx
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
COMMAND  PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
nginx   4930     root    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4931 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4931 www-data   12u  IPv4  92738      0t0  TCP localhost:http->localhost:46214 (CLOSE_WAIT)
nginx   4931 www-data   13u  IPv4  93029      0t0  TCP localhost:http->localhost:47053 (CLOSE_WAIT)
nginx   4931 www-data   14u  IPv4  93033      0t0  TCP localhost:http->localhost:47070 (CLOSE_WAIT)
nginx   4931 www-data   15u  IPv4  94746      0t0  TCP localhost:http->localhost:47071 (CLOSE_WAIT)
nginx   4931 www-data   16u  IPv4  94750      0t0  TCP localhost:http->localhost:47072 (CLOSE_WAIT)
nginx   4931 www-data   17u  IPv4  94777      0t0  TCP localhost:http->localhost:47089 (CLOSE_WAIT)
nginx   4931 www-data   18u  IPv4  94779      0t0  TCP localhost:http->localhost:47090 (CLOSE_WAIT)
nginx   4931 www-data   19u  IPv4  96173      0t0  TCP localhost:http->localhost:47091 (CLOSE_WAIT)
nginx   4931 www-data   20u  IPv4  93050      0t0  TCP localhost:http->localhost:47092 (CLOSE_WAIT)
nginx   4932 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4934 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)
nginx   4935 www-data    6u  IPv4  92455      0t0  TCP *:http (LISTEN)

Tested under Ubuntu 12.04, 14.04, 16.04 x86_64.

Attachments (1)

mod_test_wait_ngx.tar.bz2​ (1.2 KB ) - added by Dmitry Utkin 8 years ago.
module to reproduce the WAIT_CLOSE problem

Download all attachments as: .zip

Change History (5)

by Dmitry Utkin, 8 years ago

Attachment: mod_test_wait_ngx.tar.bz2​ added

module to reproduce the WAIT_CLOSE problem

comment:1 by Maxim Dounin, 8 years ago

Resolution: → invalid
Status: new → closed

This is due to a bug in the module provided.

in reply to:  1 comment:2 by Dmitry Utkin, 8 years ago

Replying to mdounin:

This is due to a bug in the module provided.

Can you please explain what's wrong? Or provide minimal working module?

Last edited 8 years ago by Dmitry Utkin (previous) (diff)

comment:3 by Maxim Dounin, 8 years ago

You can see a working example in ​src/http/modules/ngx_http_dav_module.c. The key difference is that after ngx_http_read_client_request_body() you are responsible to call ngx_http_finalize_request() in the body handler. Also, your compiler should tell you that you are passing a wrong function to ngx_http_read_client_request_body().

Version 0, edited 8 years ago by Maxim Dounin (next)

comment:4 by Dmitry Utkin, 8 years ago

Thank you very much for your answer! This solved my problem.

Note: See TracTickets for help on using tickets.