Opened 5 years ago
Last modified 5 years ago
#1950 closed defect
nginx/1.17.9 randomly truncating some large proxy responses — at Version 1
| Reported by: | Artem Russakovskii | Owned by: | |
|---|---|---|---|
| Priority: | major | Milestone: | |
| Component: | nginx-core | Version: | |
| Keywords: | truncate, truncation, cut off, response | Cc: | |
| uname -a: | Linux 5.4.10-x86_64-linode132 #1 SMP PREEMPT Thu Jan 9 21:17:12 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | ||
| nginx -V: |
nginx/1.17.9
built by gcc 7.4.0 (SUSE Linux) built with OpenSSL 1.1.0i-fips 14 Aug 2018 (running with OpenSSL 1.1.1f 31 Mar 2020) TLS SNI support enabled configure arguments: --prefix=/usr/ --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/run/nginx.pid --lock-path=/run/nginx.lock --http-client-body-temp-path=/var/lib/nginx/tmp/ --http-proxy-temp-path=/var/lib/nginx/proxy/ --http-fastcgi-temp-path=/var/lib/nginx/fastcgi/ --http-uwsgi-temp-path=/var/lib/nginx/uwsgi/ --http-scgi-temp-path=/var/lib/nginx/scgi/ --user=nginx --group=nginx --without-select_module --without-poll_module --with-threads --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-perl=/usr/bin/perl --with-mail=dynamic --with-mail_ssl_module --with-stream=dynamic --with-stream_ssl_module --with-stream_realip_module --with-stream_ssl_preread_module --with-pcre --with-pcre-jit --with-libatomic --with-compat --add-module=nginx_upstream_check_module-0.3.0 --add-dynamic-module=ngx-fancyindex-0.4.2 --add-dynamic-module=headers-more-nginx-module-0.33 --add-dynamic-module=nginx-rtmp-module-1.2.1 --add-dynamic-module=ngx_http_geoip2_module-3.3 --with-cc-opt='-fmessage-length=0 -grecord-gcc-switches -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -g -fPIC -D_GNU_SOURCE' --with-ld-opt='-Wl,-z,relro,-z,now -pie' |
||
Description (last modified by )
Hi,
We're using nginx/1.17.9 on 5 servers, and one of them started behaving oddly about a week ago, randomly truncating some proxy responses - at least the ones we noticed are specifically WordPress xmlrpc responses from Apache running on the same host, which nginx proxies to.
A few things stand out about this.
- We haven't noticed any other data getting truncated similarly, only these xmlrpc XML API responses (at least not yet).
- After we narrowed it down to nginx (querying Apache directly produced no truncation), I decided to first restart nginx. After the restart, the issue went away (at least for now). None of the other 4 web servers running nginx and apache in the same manner have ever exhibited this behavior.
- The machine has plenty of space (1.3TB), so it's not an issue of lack of space.
- Here's the most interesting bit. This machine was rebooted about a week ago, which is exactly when the issue started.
The 2nd interesting bit is the reboot kicked in a new kernel: 5.4.10-x86_64-linode132.
All the other machines haven't been rebooted yet and are running 5.1.17-x86_64-linode128.
- The response is about 600KB. Sometimes ~500KB was returned, sometimes less (~400KB). Sometimes the full response was there.
Is this a known issue? Do you have any ideas about why it may have happened?
Note:
See TracTickets
for help on using tickets.
