id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc	uname	nginx_version
353	nginx 1.4.[0|1] empty reply from server	Maximilian Weber		"
Problem:

curl: random curl: (52) Empty reply from server 
chrome: random Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection without sending any data.

Upon upgrading to the new nginx version we saw random error messages when doing our healthchecks to static files on the server. this isn't utilizing passenger at this point but plain serving static files
- this is only happening when during high traffic. 
- there is no loadbalancer involved. 
- error.log with debug doesn't show anything.
- error shows with disabled iptables 
- no selinux

empty reply:

14:57:07.231413 IP 178.210.115.51.59960 > app515.blah.com.http: Flags [S], seq 2421292407, win 14600, options [mss 1460,sackOK,TS val 92501540 ecr 0,nop,wscale 7], length 0
14:57:07.231447 IP app515.blah.com.http > 178.210.115.51.59960: Flags [S.], seq 1123747008, ack 2421292408, win 14480, options [mss 1460,sackOK,TS val 1552315767 ecr 92501540,nop,wscale 7], length 0
14:57:07.250915 IP 178.210.115.51.59960 > app515.blah.com.http: Flags [.], ack 1, win 115, options [nop,nop,TS val 92501545 ecr 1552315767], length 0
14:57:07.250973 IP app515.blah.com.http > 178.210.115.51.59960: Flags [F.], seq 1, ack 1, win 115, options [nop,nop,TS val 1552315855 ecr 92501545], length 0
14:57:07.251429 IP 178.210.115.51.59960 > app515.blah.com.http: Flags [P.], seq 1:95, ack 1, win 115, options [nop,nop,TS val 92501545 ecr 1552315767], length 94
14:57:07.251448 IP app515.blah.com.http > 178.210.115.51.59960: Flags [R], seq 1123747009, win 0, length 0
14:57:07.270451 IP 178.210.115.51.59960 > app515.blah.com.http: Flags [F.], seq 95, ack 2, win 115, options [nop,nop,TS val 92501549 ecr 1552315855], length 0
14:57:07.270474 IP app515.blah.com.http > 178.210.115.51.59960: Flags [R], seq 1123747010, win 0, length 0
14:57:07.670518 IP 178.210.115.51.54293 > app515.blah.com.http: Flags [F.], seq 13924, ack 2174, win 16276, length 0
14:57:07.674907 IP app515.blah.com.http > 178.210.115.51.54293: Flags [F.], seq 2174, ack 13925, win 334, length 0
14:57:07.702982 IP 178.210.115.51.54293 > app515.blah.com.http: Flags [.], ack 2175, win 16276, length 0
14:57:09.293112 IP 178.210.115.51.53615 > app515.blah.com.http: Flags [F.], seq 17172, ack 4741, win 16425, length 0
14:57:09.293153 IP app515.blah.com.http > 178.210.115.51.53615: Flags [F.], seq 4741, ack 17173, win 390, length 0
14:57:09.338779 IP 178.210.115.51.53615 > app515.blah.com.http: Flags [.], ack 4742, win 16425, length 0


good reply:
14:57:54.336213 IP 178.210.115.51.59975 > app515.blah.com.http: Flags [S], seq 1550226949, win 14600, options [mss 1460,sackOK,TS val 92513316 ecr 0,nop,wscale 7], length 0
14:57:54.336236 IP app515.blah.com.http > 178.210.115.51.59975: Flags [S.], seq 1532621207, ack 1550226950, win 14480, options [mss 1460,sackOK,TS val 1552362940 ecr 92513316,nop,wscale 7], length 0
14:57:54.544906 IP 178.210.115.51.59975 > app515.blah.com.http: Flags [.], ack 1, win 115, options [nop,nop,TS val 92513368 ecr 1552362940], length 0
14:57:54.545233 IP 178.210.115.51.59975 > app515.blah.com.http: Flags [P.], seq 1:95, ack 1, win 115, options [nop,nop,TS val 92513368 ecr 1552362940], length 94
14:57:54.545257 IP app515.blah.com.http > 178.210.115.51.59975: Flags [.], ack 95, win 114, options [nop,nop,TS val 1552363149 ecr 92513368], length 0
14:57:54.546194 IP app515.blah.com.http > 178.210.115.51.59975: Flags [P.], seq 1:231, ack 95, win 114, options [nop,nop,TS val 1552363150 ecr 92513368], length 230
14:57:54.749511 IP 178.210.115.51.59975 > app515.blah.com.http: Flags [.], ack 231, win 123, options [nop,nop,TS val 92513419 ecr 1552363150], length 0
14:57:54.750097 IP 178.210.115.51.59975 > app515.blah.com.http: Flags [F.], seq 95, ack 231, win 123, options [nop,nop,TS val 92513419 ecr 1552363150], length 0
14:57:54.750135 IP app515.blah.com.http > 178.210.115.51.59975: Flags [F.], seq 231, ack 96, win 114, options [nop,nop,TS val 1552363354 ecr 92513419], length 0
14:57:54.857239 IP 178.210.115.51.59975 > app515.blah.com.http: Flags [.], ack 232, win 123, options [nop,nop,TS val 92513446 ecr 1552363354], length 0
"	defect	closed	minor		nginx-core	1.3.x	fixed	empty reply from server		Linux app519.blah.com 2.6.32-220.el6.x86_64 #1 SMP Sat Dec 10 17:04:11 CST 2011 x86_64 x86_64 x86_64 GNU/Linux	"nginx version: nginx/1.4.1
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) 
TLS SNI support enabled
configure arguments: --user=nginx --group=nginx --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --pid-path=/var/run/nginx.pid --lock-path=/var/lock/subsys/nginx --with-http_stub_status_module --with-http_realip_module --with-http_ssl_module --with-http_gzip_static_module --with-mail --with-mail_ssl_module --with-ipv6 --add-module=/usr/lib/ruby/gems/1.8/gems/passenger-enterprise-server-4.0.0.rc4/ext/nginx"