"Vary: X-Forwarded-Proto" should be removed

[shaula@…]

I'm using Nginx as SSL terminator and I've got the following setup:

Internet --> Nginx:443 --> Varnish:6080 --> Apache:80

Details

Doing this I use this as part of my config:

location / {
            proxy_pass http://127.0.0.1:6080;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header Host $host;
}

While Apache sees the request header "X-Forwarded-Proto: https" it adds a response header "Vary: X-Forwarded-Proto".
Depending on the resource it might also send a "Vary: X-Forwarded-Proto, Accept-Encoding".

Now this Vary header is useful, because Varnish can create different cache entries for pages that are HTTPS over HTTP and another one for plain-HTTP.

Request for change

As Nginx runs as SSL terminator it should remove the response header value "X-Forwarded-Proto" from the "Vary" header! In some cases other Vary values such as "Accept-Encoding" should be kept.

Reasoning

Nginx terminated SSL, so we tell all services behind that this request was orignally an HTTPS one by using "X-Forwarded-Proto: https". As this request header was not present on the outside (Internet), it makes no sense to keep "Vary: X-Forwarded-Proto".

Effects

Keeping a "Vary: X-Forwarded-Proto" causes cache malfunctioning in IE9+ (see also: ​https://blogs.msdn.microsoft.com/ieinternals/2009/06/17/vary-with-care/). IE refuses to cache stuff with "custom" Vary values. In addition it refuses to use webfonts, e.g. woff2 files, served with this response header.

Side Note

If Apache is used as SSL terminator it does exactly this.

[Maxim Dounin]

While Apache sees the request header "X-Forwarded-Proto: https" it adds a response header "Vary: X-Forwarded-Proto".

I don't see such a behaviour in the Apache sources. It is probably something you've configured your Apache server to do. E.g., a RewriteCond which references a header will add the header to the Vary returned.

If Apache is used as SSL terminator it does exactly this.

Same here, I don't see anything like this in the Apache code.

I don't see any problems with nginx behaviour here. It's up to the backend to decide which headers to return. If it doesn't work for some reason (because using Vary is unwise in general, or because IE doesn't like it) - it may be a good idea to change what the backend returns. If you can't or don't want to change your backend for some reason, you can do anything you want with the Vary header on the nginx side manually, using the ​proxy_hide_header and ​add_header directives.

Making such Vary modifications easier may qualify as a feature request, though I doubt anybody will look into it. It looks easy enough to do with existing mechanisms when needed.

[shaula@…]

Thanks for the hint with ​RewriteCond. I'm sorry that I mislead you with Apache's behavior.

Unfortunately Varnish and other caches in between may need Vary: X-Forwarded-Proto, so the backend has to send this header. On the other hand no cache on the outside would need it, as there is no request header X-Forwarded-Proto: https.

I appreciate your answer and having a workaround with core utilities (i.e. without external modules) would be fine for now.

Proposed workaround

you can do anything you want with the Vary header on the nginx side manually, using the ​proxy_hide_header and ​add_header directives.

I wish it was so easy. Unfortunately the proxy_hide_header removes all header values. This means it works only for the first use case:

Use Case 1: Vary: X-Forwarded-Proto --> no header
Use Case 2: Vary: X-Forwarded-Proto, Accept-Encoding --> Vary: Accept-Encoding

My trial & error for a solution

Using these directives in the location / block did result in failure (Vary header is removed completely and does not show up again):

proxy_hide_header Vary;
gzip_vary on;

Another try using if (is evil, I know) and it did not work as well:

if ($upstream_http_vary ~* "Accept-Encoding") {
    add_header X-Vary "Accept-Encoding";
}

Unfortunately I'm missing the LUA module, otherwise I'd have been able to do: ​http://stackoverflow.com/a/36236045

Possible solution
I got the Perl module available, but failed to find a good example of header string manipulation. Probably I'd have to dig into ​ngx_http_perl_module.

If there is an easier solution, please give me some hints.

[Maxim Dounin]

Replying to shaula@…:

Unfortunately Varnish and other caches in between may need Vary: X-Forwarded-Proto, so the backend has to send this header.

I was trying to say above that it may be a good idea to redesign your system to avoid the need. But if you can't or don't want to for some reason, see below.

On the other hand no cache on the outside would need it, as there is no request header X-Forwarded-Proto: https.

The problem is that this is something you know, but not nginx.

you can do anything you want with the Vary header on the nginx side manually, using the ​proxy_hide_header and ​add_header directives.

I wish it was so easy. Unfortunately the proxy_hide_header removes all header values.

The original value of the header as got from upstream is available via the $upstream_http_vary variable, even if you hide it. That is, something like this can be used to preserve Accept-Encoding in the Vary, but strip anything else:

map $upstream_http_vary $vary {
        default "";
        ~Accept-Encoding "Accept-Encoding";
}

add_header Vary $vary;
proxy_hide_header Vary;

[shaula@…]

Thanks for your great support and pointing out the ​map directive. I never stumbled upon it.

As I needed to allow arbitrary value combinations of User-Agent, Accept-Encoding, Origin, etc. I tried solving the problem with regular expressions. I ended up using the following configuration within the http block:

    map $upstream_http_vary $vary {
        default "";

        # use case 1 - none:                ""
        # use case 2 - nowhere, but others: "Vary: Accept-Encoding"
        # use case 3 - standalone:          "Vary: X-Forwarded-Proto"
        # use case 4 - at the beginning:    "Vary: X-Forwarded-Proto,Accept-Encoding"
        # use case 5 - in the middle:       "Vary: Accept-Encoding,X-Forwarded-Proto,User-Agent"
        # use case 6 - at the end:          "Vary: User-Agent,X-Forwarded-Proto"

        # 1: trivial

        # 3: use empty value
        ~*^X-Forwarded-Proto$ "";

        # 4: use rest
        ~*^X-Forwarded-Proto,(?<after>.*)$ "$after";

        # 5: middle - FIXME - how?

        # 6: use beginning
        ~*^(?<before>.*),X-Forwarded-Proto$ "$before";

        # 2: use everything
        ~*^(?<all>.*)$ "$all";
    }

    add_header Vary $vary;
    proxy_hide_header Vary;

I succeeded for 5/6 use cases.

Open Issue

The one failing is number 5, where I tried to use these non-working regex/var combinations to get the unwanted value away from the middle:

~^(?<before>.*),X-Forwarded-Proto,(?<after>.*)$ "$before$after";

~^(?<before>.*),X-Forwarded-Proto,(?<after>.*)$ "${before}${after}";

...but it fails as only one variable seems to be allowed. Is there any way to workaround that remaining issue (which might not occur very often - but you never know).

[Maxim Dounin]

Multiple variables in the resulting value in map can be used starting with nginx 1.11.0. In previous versions you can use multiple variables directly in the add_header directive.

[Maxim Dounin]

It doesn't looks like there is anything here which needs to be implemented in nginx. As outlined above, proper solution is to avoid the wrong header being returned by the backend. If for some reason this is not possible, existing configuration mechanisms are enough to modify the header.