Opened 2 years ago

Last modified 6 months ago

#1423 accepted defect

response vary headers not used in the cache key

Reported by: gedl@… Owned by:
Priority: minor Milestone:
Component: other Version: 1.12.x
Keywords: Cc:
uname -a: Darwin hostname.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
nginx -V: nginx version: nginx/1.12.2 built by clang 9.0.0 (clang-900.0.38) built with OpenSSL 1.0.2l 25 May 2017 (running with OpenSSL 1.0.2m 2 Nov 2017) TLS SNI support enabled configure arguments: --prefix=/usr/local/Cellar/nginx/1.12.2_1 --with-http_ssl_module --with-pcre --sbin-path=/usr/local/Cellar/nginx/1.12.2_1/bin/nginx --with-cc-opt='-I/usr/local/opt/pcre/include -I/usr/local/opt/openssl/include' --with-ld-opt='-L/usr/local/opt/pcre/lib -L/usr/local/opt/openssl/lib' --conf-path=/usr/local/etc/nginx/nginx.conf --pid-path=/usr/local/var/run/nginx.pid --lock-path=/usr/local/var/run/nginx.lock --http-client-body-temp-path=/usr/local/var/run/nginx/client_body_temp --http-proxy-temp-path=/usr/local/var/run/nginx/proxy_temp --http-fastcgi-temp-path=/usr/local/var/run/nginx/fastcgi_temp --http-uwsgi-temp-path=/usr/local/var/run/nginx/uwsgi_temp --http-scgi-temp-path=/usr/local/var/run/nginx/scgi_temp --http-log-path=/usr/local/var/log/nginx/access.log --error-log-path=/usr/local/var/log/nginx/error.log --with-http_gzip_static_module --with-http_v2_module

Description

Scenario:
use nginx to cache server responses based on

  • scheme of the request
  • host of the proxy
  • request uri
  • values of the request headers as selected by the response vary headers

Request 1:

curl -H "Authorization: one" -I http://localhost:8081/some_uri
HTTP/1.1 200 OK
Server: nginx
Date: Sat, 11 Nov 2017 01:56:24 GMT
Content-Type: application/json
Content-Length: 1389
Connection: keep-alive
Access-Control-Allow-Headers: Accept, Authorization, Content-Type, x-api-key, Accept-Language
Access-Control-Allow-Methods: GET, POST, PUT, PATCH, OPTIONS, DELETE
Access-Control-Allow-Origin: *
Vary: Authorization
Cache-Control: no-transform, max-age=10, s-maxage=10
request-id: da208a4310e0a67f
loc: local
processing-time: 307
Vary: Accept-Encoding
X-Cached: MISS

Comment: the request was a cache miss. all good

Request 2

 curl -H "Authorization: two" -I http://localhost:8081/api/some_uri
HTTP/1.1 200 OK
Server: nginx
Date: Sat, 11 Nov 2017 01:56:28 GMT
Content-Type: application/json
Content-Length: 1389
Connection: keep-alive
Access-Control-Allow-Headers: Accept, Authorization, Content-Type, x-api-key, Accept-Language
Access-Control-Allow-Methods: GET, POST, PUT, PATCH, OPTIONS, DELETE
Access-Control-Allow-Origin: *
Vary: Authorization
Cache-Control: no-transform, max-age=10, s-maxage=10
request-id: da208a4310e0a67f
loc: local
processing-time: 307
Vary: Accept-Encoding
X-Cached: HIT

Comment: a different value for the authorization header was specified on the second request. As the response includes Vary: Authorization, I'd expect the second request to be a cache MISS. However, it was a cache HIT

Configuration file:

worker_processes 4;


events {
        worker_connections 768;
        multi_accept on;
}

http {

        upstream target {
                server       localhost:8082;
        }

        proxy_cache_path /Users/gluiz/dev/nginx/cache levels=1:2 keys_zone=my_cache:10m max_size=10g inactive=60m use_temp_path=off;
        add_header X-Cached $upstream_cache_status;


server {
        listen       8081;
        server_name  anonymous;
        proxy_intercept_errors off;
        location / {
                 proxy_cache my_cache;
                proxy_set_header Host $http_host;
                proxy_pass http://target$request_uri;
        }

        ssl_verify_client off;
}



        server_tokens off;

}

Change History (6)

comment:1 Changed 2 years ago by gedl@…

Note: it appears that the problem may be related to the presence of multiple Vary headers in the response. This is RFC accepted, and the semantics is the same as one Vary header with all the values:

Vary: a
Vary: b

is semantically equivalent to:

Vary: a, b

comment:2 Changed 2 years ago by mdounin

  • Priority changed from major to minor
  • Status changed from new to accepted

Strictly speaking, this not something allowed by RFC. Quoting RFC 7230:

   A sender MUST NOT generate multiple header fields with the same field
   name in a message unless either the entire field value for that
   header field is defined as a comma-separated list [i.e., #(values)]
   or the header field is a well-known exception (as noted below).

And the Vary header field is defined as follows, see RFC 7231:

     Vary = "*" / 1#field-name

That is, "the entire field value for that header field is defined as a comma-separated list" is not true due to "*", so sender is not allowed to use multiple Vary headers.

On the other hand, the intention is clear enough, and we probably should either respect all the Vary headers, or simply disable caching in such a case.

comment:3 Changed 2 years ago by gedl@…

Yes, agreed.

It'd be much more interesting to respect all Vary headers. As a workaround, is there a way of normalising the headers prior to the calculation of the cache key?

comment:4 Changed 12 months ago by gedl@…

Hi

Was there any movement on this over the past year?

comment:5 Changed 11 months ago by mdounin

Was there any movement on this over the past year?

I haven't seen any patches submitted to address this.

comment:6 Changed 6 months ago by martijn.zijlstra@…

I cant see your used proxy_cache_key so i assume you're using the default :

proxy_cache_key "$scheme$proxy_host$uri$is_args$args";

Changing it to should solve your issue :

proxy_cache_key "$scheme$proxy_host$uri$is_args$args$http_authorization";
Note: See TracTickets for help on using tickets.