Opened 8 years ago

Closed 8 years ago

#104 closed defect (invalid)

Duplicate FCGI_BEGIN_REQUEST records with fastcgi_keep_conn on

Reported by: scott.conger@… Owned by: somebody
Priority: minor Milestone:
Component: nginx-core Version: 1.1.x
Keywords: Cc:
uname -a: Linux eng004.brandnet.local 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.1.14
built by gcc 4.1.2 20080704 (Red Hat 4.1.2-50)
configure arguments: --prefix=. --without-http_rewrite_module --with-zlib=/home/BRANDNET/sconger/temp/zlib-1.2.6/

Description

When experimenting with the 'fastcgi_keep_conn' option, I'm seeing a periodic issue with getting two records identical FCGI_BEGIN_REQUEST records with the requestIdB1 and requestIdB0 bytes set to the same value on the same connection.

The logging on our application side shows this:

Conn 1 Duplicate request ID (1) received from Web Server. Request ignored.
Conn 1 Record: FastCGIRecord[type=BEGIN_REQUEST,requestID=1,contentLength=8,content=<empty>]
Last processed: FastCGIRecord[type=BEGIN_REQUEST,requestID=1,contentLength=8,content=<empty>]

Obviously, this could be a problem on our side, but I debugged for quite a while and it looks like we are receiving a duplicate record. We have successfully used fast CGI with persistent connection with another web server.

This showed up when running a test program that sent 500,000 test GET requests. It only happens 5 to 8 times in such a test. The practical side effect is that those 5 to 8 GET requests time out as we're ignoring the seemingly duplicate FCGI_BEGIN_REQUEST record.

I don't see anything getting logged in nginx's logs when the issue appears and we don't see any issues if we comment out 'fastcgi_keep_conn on;' in the configuration.

Attachments (1)

nginx.conf (8.0 KB ) - added by scott.conger@… 8 years ago.
Main configuration file (GET requests sent to /charon/adrequest)

Download all attachments as: .zip

Change History (5)

by scott.conger@…, 8 years ago

Attachment: nginx.conf added

Main configuration file (GET requests sent to /charon/adrequest)

comment:1 by Maxim Dounin, 8 years ago

As nginx doesn't use fastcgi request multiplexing, it always uses requestId 1. So the question is if nginx sends another FCGI_BEGIN_REQUEST on the same connection before your app sends FCGI_END_REQUEST. The nginx code suggests it should never happen, as connections are only subject to reuse after FCGI_END_REQUEST is received.

You may try logging all traffic between nginx and your app e.g. with "tcpdump -s0 -w file ..." to see what actually happens on the wire (and make sure your app logs enough data to identify particular connection it thinks it got duplicate FCGI_BEGIN_REQUEST). This should make it clear if the problem is in nginx or in your app.

comment:2 by scott.conger@…, 8 years ago

Thanks for the exceptionally quick response Maxim. I'll investigate and get back to you. It might take a few days due to other work.

comment:3 by scott.conger@…, 8 years ago

I traced the issue to a race condition in our fast CGI library. I've fixed the issue there and it's working fine. Please close bug report.

comment:4 by Maxim Dounin, 8 years ago

Resolution: invalid
Status: newclosed
Note: See TracTickets for help on using tickets.