Opened 6 years ago

Closed 22 months ago

#1423 closed defect (fixed)

response vary headers not used in the cache key

Reported by: gedl@… Owned by:
Priority: minor Milestone:
Component: other Version: 1.12.x
Keywords: Cc:
uname -a: Darwin hostname.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
nginx -V: nginx version: nginx/1.12.2
built by clang 9.0.0 (clang-900.0.38)
built with OpenSSL 1.0.2l 25 May 2017 (running with OpenSSL 1.0.2m 2 Nov 2017)
TLS SNI support enabled
configure arguments: --prefix=/usr/local/Cellar/nginx/1.12.2_1 --with-http_ssl_module --with-pcre --sbin-path=/usr/local/Cellar/nginx/1.12.2_1/bin/nginx --with-cc-opt='-I/usr/local/opt/pcre/include -I/usr/local/opt/openssl/include' --with-ld-opt='-L/usr/local/opt/pcre/lib -L/usr/local/opt/openssl/lib' --conf-path=/usr/local/etc/nginx/nginx.conf --pid-path=/usr/local/var/run/nginx.pid --lock-path=/usr/local/var/run/nginx.lock --http-client-body-temp-path=/usr/local/var/run/nginx/client_body_temp --http-proxy-temp-path=/usr/local/var/run/nginx/proxy_temp --http-fastcgi-temp-path=/usr/local/var/run/nginx/fastcgi_temp --http-uwsgi-temp-path=/usr/local/var/run/nginx/uwsgi_temp --http-scgi-temp-path=/usr/local/var/run/nginx/scgi_temp --http-log-path=/usr/local/var/log/nginx/access.log --error-log-path=/usr/local/var/log/nginx/error.log --with-http_gzip_static_module --with-http_v2_module

Description

Scenario:
use nginx to cache server responses based on

  • scheme of the request
  • host of the proxy
  • request uri
  • values of the request headers as selected by the response vary headers

Request 1:

curl -H "Authorization: one" -I http://localhost:8081/some_uri
HTTP/1.1 200 OK
Server: nginx
Date: Sat, 11 Nov 2017 01:56:24 GMT
Content-Type: application/json
Content-Length: 1389
Connection: keep-alive
Access-Control-Allow-Headers: Accept, Authorization, Content-Type, x-api-key, Accept-Language
Access-Control-Allow-Methods: GET, POST, PUT, PATCH, OPTIONS, DELETE
Access-Control-Allow-Origin: *
Vary: Authorization
Cache-Control: no-transform, max-age=10, s-maxage=10
request-id: da208a4310e0a67f
loc: local
processing-time: 307
Vary: Accept-Encoding
X-Cached: MISS

Comment: the request was a cache miss. all good

Request 2

 curl -H "Authorization: two" -I http://localhost:8081/api/some_uri
HTTP/1.1 200 OK
Server: nginx
Date: Sat, 11 Nov 2017 01:56:28 GMT
Content-Type: application/json
Content-Length: 1389
Connection: keep-alive
Access-Control-Allow-Headers: Accept, Authorization, Content-Type, x-api-key, Accept-Language
Access-Control-Allow-Methods: GET, POST, PUT, PATCH, OPTIONS, DELETE
Access-Control-Allow-Origin: *
Vary: Authorization
Cache-Control: no-transform, max-age=10, s-maxage=10
request-id: da208a4310e0a67f
loc: local
processing-time: 307
Vary: Accept-Encoding
X-Cached: HIT

Comment: a different value for the authorization header was specified on the second request. As the response includes Vary: Authorization, I'd expect the second request to be a cache MISS. However, it was a cache HIT

Configuration file:

worker_processes 4;


events {
        worker_connections 768;
        multi_accept on;
}

http {

        upstream target {
                server       localhost:8082;
        }

        proxy_cache_path /Users/gluiz/dev/nginx/cache levels=1:2 keys_zone=my_cache:10m max_size=10g inactive=60m use_temp_path=off;
        add_header X-Cached $upstream_cache_status;


server {
        listen       8081;
        server_name  anonymous;
        proxy_intercept_errors off;
        location / {
                 proxy_cache my_cache;
                proxy_set_header Host $http_host;
                proxy_pass http://target$request_uri;
        }

        ssl_verify_client off;
}



        server_tokens off;

}

Change History (8)

comment:1 by gedl@…, 6 years ago

Note: it appears that the problem may be related to the presence of multiple Vary headers in the response. This is RFC accepted, and the semantics is the same as one Vary header with all the values:

Vary: a
Vary: b

is semantically equivalent to:

Vary: a, b

comment:2 by Maxim Dounin, 6 years ago

Priority: majorminor
Status: newaccepted

Strictly speaking, this not something allowed by RFC. Quoting RFC 7230:

   A sender MUST NOT generate multiple header fields with the same field
   name in a message unless either the entire field value for that
   header field is defined as a comma-separated list [i.e., #(values)]
   or the header field is a well-known exception (as noted below).

And the Vary header field is defined as follows, see RFC 7231:

     Vary = "*" / 1#field-name

That is, "the entire field value for that header field is defined as a comma-separated list" is not true due to "*", so sender is not allowed to use multiple Vary headers.

On the other hand, the intention is clear enough, and we probably should either respect all the Vary headers, or simply disable caching in such a case.

comment:3 by gedl@…, 6 years ago

Yes, agreed.

It'd be much more interesting to respect all Vary headers. As a workaround, is there a way of normalising the headers prior to the calculation of the cache key?

comment:4 by gedl@…, 5 years ago

Hi

Was there any movement on this over the past year?

comment:5 by Maxim Dounin, 5 years ago

Was there any movement on this over the past year?

I haven't seen any patches submitted to address this.

comment:6 by martijn.zijlstra@…, 5 years ago

I cant see your used proxy_cache_key so i assume you're using the default :

proxy_cache_key "$scheme$proxy_host$uri$is_args$args";

Changing it to should solve your issue :

proxy_cache_key "$scheme$proxy_host$uri$is_args$args$http_authorization";

comment:7 by Maxim Dounin <mdounin@…>, 22 months ago

In 8035:cd73509f21e2/nginx:

Upstream: handling of multiple Vary headers (ticket #1423).

Previously, only the last header value was used when caching.

comment:8 by Maxim Dounin, 22 months ago

Resolution: fixed
Status: acceptedclosed

Fixed, thanks.

Note: See TracTickets for help on using tickets.