gunzip module works inappropriately

[Alex Zhang]

Hi, everyone!

So i have a backend server, which proxied by nginx.

I found the gunzip module works inappropriately when the request method is HEAD and the backend server returns the gzipped data but with the Content-Length header instead of Tranfer-Encoding.

The backend server's output is followed(some irrelevant headers removed by myself).

HTTP/1.1 200 OK
Date: Wed, 26 Jul 2017 04:42:59 GMT
Content-Type: text/plain
Content-Length: 83
Connection: keep-alive
Accept-Ranges: bytes
Content-Encoding: gzip
Content-Transfer-Encoding: binary
Last-Modified: Tue, 25 Jul 2017 08:32:49 GMT
Vary: Accept-Encoding

When i enable the gunzip, nginx gunzip module will remove the Content-Length header, and the chunked filter module also will be bypassed in this scene. Ultimately, we will see a HTTP response without Conent-Length and Tranfer-Encoding:

curl -I http://127.0.0.1:9091/ios/1500971336_524/build_info.txt\?v\=7623874234
HTTP/1.1 200 OK
Server: nginx/1.13.4
Date: Tue, 25 Jul 2017 05:34:38 GMT
Content-Type: text/plain
Connection: keep-alive
Cache-Control: public, max-age=31536000
Content-Transfer-Encoding: binary
Last-Modified: Tue, 25 Jul 2017 08:32:49 GMT
Vary: Accept-Encoding

And my nginx proxy config is followed:

server {
    listen 9091;
    location / {
        gunzip on;

        proxy_set_header Accept-Encoding gzip;
        proxy_set_header Host hotupdatehlfmj.tfhygame.com;
        proxy_pass http://backend;
    }
}

upstream backend {
    server 1.1.1.1:80 max_fails=3 fail_timeout=30;
}

I think the gunzip module should ignore this scene(e.g. bypass the gunzip when r->request_method is HEAD) to ensure the client can get Content-Length header.

I am sorry if there is any place that does not conform to the ticket rules.

[Valentin V. Bartenev]

Ultimately, we will see a HTTP response without Conent-Length and Tranfer-Encoding

That is OK according to HTTP RFC.

I think the gunzip module should ignore this scene(e.g. bypass the gunzip when r->request_method is HEAD) to ensure the client can get Content-Length header.

Why do you think so? The Content-Length is unknown in this case.

[Alex Zhang]

I think the gunzip module should ignore this scene(e.g. bypass the gunzip when r->request_method is HEAD) to ensure the client can get Content-Length header.

Why do you think so? The Content-Length is unknown in this case.

I mean the Content-Length returned by backend.

[Alex Zhang]

Also, i think nginx shouldn't remove it.

[Valentin V. Bartenev]

Replying to tokers@…:

I think the gunzip module should ignore this scene(e.g. bypass the gunzip when r->request_method is HEAD) to ensure the client can get Content-Length header.

Why do you think so? The Content-Length is unknown in this case.

I mean the Content-Length returned by backend.

The Content-Length returned by backend is not valid for nginx response, because after unpacking the content will have different size.

[Alex Zhang]

Replying to vbart:

Replying to tokers@…:

I think the gunzip module should ignore this scene(e.g. bypass the gunzip when r->request_method is HEAD) to ensure the client can get Content-Length header.

Why do you think so? The Content-Length is unknown in this case.

I mean the Content-Length returned by backend.

The Content-Length returned by backend is not valid for nginx response, because after unpacking the content will have different size.

Ok, i see, but actually the gunzip body filter doesn't work because now the request method is HEAD, so why the gunzip header filter just bypass this scene.

[Valentin V. Bartenev]

Replying to tokers@…:

Replying to vbart:

Replying to tokers@…:

I think the gunzip module should ignore this scene(e.g. bypass the gunzip when r->request_method is HEAD) to ensure the client can get Content-Length header.

Why do you think so? The Content-Length is unknown in this case.

I mean the Content-Length returned by backend.

The Content-Length returned by backend is not valid for nginx response, because after unpacking the content will have different size.

Ok, i see, but actually the gunzip body filter doesn't work because now the request method is HEAD, so why the gunzip header filter just bypass this scene.

Because it's exactly what is the HEAD method for. Here is a quote from RFC:

The server SHOULD send the same header fields in response to a HEAD request as it would have sent if the request had been a GET, except that the payload header fields (Section 3.3) MAY be omitted.

So the gunzip filter removes the original Content-Length as like it would remove it for the GET request.

[Alex Zhang]

Thanks for your answer!

[Maxim Dounin]

As already explained by Valentin, everything works as intended.