Opened 8 years ago

Last modified 17 months ago

#1182 new enhancement

Responses with "no-cache" or "max-age=0" should be cached

Reported by: geoff.addepar.com@… Owned by:
Priority: minor Milestone:
Component: other Version: 1.9.x
Keywords: Cc:
uname -a: Linux geoff-XPS-8300 4.4.0-57-generic #78-Ubuntu SMP Fri Dec 9 23:50:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.10.0 (Ubuntu)
built with OpenSSL 1.0.2g 1 Mar 2016
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-stream --with-stream_ssl_module --with-mail --with-mail_ssl_module --with-threads

Description

I know the summary sounds contradictory, but I believe that caching these responses is allowed, when revalidation is enabled. These responses would always need to be revalidated.

Here's the relevant part of the HTTP spec:

no-cache
       If the no-cache directive does not specify a field-name, then a
      cache MUST NOT use the response to satisfy a subsequent request
      without successful revalidation with the origin server. This
      allows an origin server to prevent caching even by caches that
      have been configured to return stale responses to client requests.

So, if "no-cache" is present, then nginx is allowed to cache the response, but must treat the cache entry as if it has already expired. The next lookup requires revalidation. I believe that "max-age=0" or "s-maxage=0" should be treated the same way, but I don't have a specific reference to the spec to justify my opinion.

We have an upstream server that is capable of returning 304 Not Modified much more quickly than it can generate the response body for a 200. We want to be able to use nginx's caching abilities to store the response body (especially to share that response body between users), but we also want to revalidate with the upstream server on every request. Right now we're working around this with "max-age=1" so that things expire quickly, but technically we want them to expire right away.

Also see "Pattern 2" on the first hit when searching "HTTP Cache Best Practices".
https://jakearchibald.com/2016/caching-best-practices/

Change History (5)

comment:1 by micah.hainline@…, 4 years ago

Any update on this? This feels like a very legitimate problem. I have a server sending the following:

Cache-Control: max-age=0, public, must-revalidate;
ETag: W/"somestring"

And I expected it to cache that response when acting properly according to spec, and revalidate it with the server when it was called again. Any validation that of what's being said here, or a workaround if this problem still hasn't been fixed in the latest version?

comment:2 by Maxim Dounin, 4 years ago

Historically, nginx does not cache responses with "max-age=0", as well as responses with Expires in the past, as such responses are effectively stale and cannot be used to respond to future requests.

Since proxy_cache_revalidate support was added, there are might be some valid use cases when caching such responses might be beneficial with proxy_cache_revalidate enabled. This probably needs further investigation.

Just for the record, one of the first discussions can be seen in this thread (in Russian).

If you want to use proxy_cache_revalidate to revalidate responses immediately, a readily available workaround is to control caching with the X-Accel-Expires: @0 header instead.

comment:3 by micah.hainline@…, 4 years ago

Trying the following didn't work.

Cache-Control: max-age=0, public, must-revalidate;
ETag: W/"somestring"
X-Accel-Expires: @0

I'm assuming that max-age is still being used to determine that nginx will not cache in this case. We could change max-age=1 or some other value, but that would be counter to the spec and break other caches. Would that be the recommended workaround for this though?

comment:4 by Maxim Dounin, 4 years ago

X-Accel-Expires: @0

My bad, should be X-Accel-Expires: @1 (which is essentially 1 second after the epoch), as plain zero will be handled as "no expiration time set" elsewhere and no caching will happen.

Additionally, this should be either before the Cache-Control header to make sure nginx wont't disable caching due to max-age=0, or the Cache-Control header should be explicitly ignored with proxy_ignore_headers.

comment:5 by mczotter.b2international.com@…, 17 months ago

Hi All,

May I ask what's the status of this issue? This is really something we would like to get implemented as described in the HTTP standard.

Our use case if that helps:

Our backend service is a Git-like revision-controlled resource server, each resource has multiple versions stored in the database and users are able to access all or some of them (through fine-grained authorization, not RBAC, true per resource access-control). The backend service is responsible for calculating the visible set of resources and their contents when serving a request. We would like to be able to use HTTP caching directives to let a proxy server cache responses for more than one user, forming a shared cache and reducing response times even further. The problem is that the server must revalidate whether the user still have access not just to the system but to the resource as well and also must revalidate whether the resource revision is still the one that's the latest version. It is also possible to access versioned and timestamped data which effectively can be cached forever but even then access should be checked. The best possible way of doing this is with the s-maxage=0 or no-cache Cache-Control header value, which would enable the proxy server to cache the content, but when accessing the cached content again due to the expiration revalidation must be performed.

What we were able to achieve is using 1 second expiration with the s-maxage=1, but unfortunately there is no way of telling NGINX to force revalidate in all cases (if there is a config for that please share it with us) so due to the nature of the shared cache users are able to access to the shared content even if they don't have access in our authorization system (which is data leakage and a security issue).

I hope this makes sense.

Regards,
Mark

Note: See TracTickets for help on using tickets.