Opened 7 years ago

Closed 18 months ago

#416 closed enhancement (fixed)

proxy_cache_use_stale run updating in new thread and serve stale data to ALL

Reported by: Steve Weber Owned by:
Priority: minor Milestone:
Component: nginx-core Version:
Keywords: Cc:
uname -a: Linux c3 3.11.0-7-generic #14-Ubuntu SMP Mon Sep 16 18:37:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.4.1 (Ubuntu)
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_realip_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-mail --with-mail_ssl_module --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-dav-ext-module --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.4.1/debian/modules/ngx_http_substitutions_filter_module

Description

it would be extremely cool if proxy_cache_use_stale had a way to serve stale data to ALL requests including the one that caused the updating.

This would really smooth out all traffic so no client would get unlucky and have to wait for a page load while updating.

Change History (11)

comment:2 by Brian Davis, 7 years ago

I agree, this would be a very useful feature to us and other nginx users. I put a bounty of $500 for it at Bountysource https://www.bountysource.com/issues/972735-proxy_cache_use_stale-run-updating-in-new-thread-and-serve-stale-data-to-all

in reply to:  2 comment:3 by Igor Kukushkin, 6 years ago

Replying to Brian Davis:

I agree, this would be a very useful feature to us and other nginx users. I put a bounty of $500 for it at Bountysource https://www.bountysource.com/issues/972735-proxy_cache_use_stale-run-updating-in-new-thread-and-serve-stale-data-to-all

Well. I had experience with writing Nginx modules as a freelancer. Going to try this one. Brian, when can I pick you on freenode's IRC tomorrow?

I think that changes to current architecture won't be so large (additional flag to enable serving stale data with backend on-line, and revalidating cache as 'updating from stale state'. Probably this additional state (together with 1-2 options) will fit right in, even without introducing dirty hacks with HTTP request being re-issued from server itself.

comment:4 by Brian Davis, 6 years ago

Hey Igor, sounds good. I'll make a point to be in the nginx channel on freenode this week between 10am and 5pm CST so we can chat. My nick is bmdavi3. Hopefully a time in there works for you.

comment:5 by Steve Weber, 6 years ago

any news on this?

comment:6 by Brian Davis, 6 years ago

Not that I know of. I never heard from Igor in IRC so that bounty is still there. However, it sounds like varnish just implemented this feature today with their 4.0.0 release. So if you already have varnish in your stack, it should be easy. We do not currently use Varnish where I work, but this feature is so nice, we are talking about adding it to our stack, just for this.

https://www.varnish-cache.org/lists/pipermail/varnish-announce/2014-April/000696.html

* Background (re)fetch of expired objects. On a cache miss where a stale copy
is available, serve the client the stale copy while fetching an updated copy
from the backend in the background.

comment:7 by Felipe Cerqueira skylazart, 6 years ago

Hi, I would like to collaborate working on this new feature.
Brian, do you have any news about it? Is it already done?
Thanks

in reply to:  7 comment:8 by Brian Davis, 6 years ago

Replying to Felipe Cerqueira skylazart:

Hi, I would like to collaborate working on this new feature.
Brian, do you have any news about it? Is it already done?
Thanks

I have no news on this. As far as I know nobody is working on it, so feel free to take a shot at it

comment:9 by richard-fairthorne@…, 4 years ago

Last response was 22 months ago. I am interested in looking at this. Is anyone else currently working on it?

comment:10 by snizovtsev@…, 18 months ago

Isn't proxy_cache_background_update directive introduced by nginx 1.11.10 (14 Feb 2017) already implements this feature?

comment:11 by Maxim Dounin, 18 months ago

Resolution: fixed
Status: newclosed

Yes, thanks.

Note: See TracTickets for help on using tickets.