Opened 6 years ago

Closed 7 months ago

#416 closed enhancement (fixed)

proxy_cache_use_stale run updating in new thread and serve stale data to ALL

Reported by: steve.r.weber@… Owned by:
Priority: minor Milestone:
Component: nginx-core Version:
Keywords: Cc:
uname -a: Linux c3 3.11.0-7-generic #14-Ubuntu SMP Mon Sep 16 18:37:27 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.4.1 (Ubuntu) TLS SNI support enabled configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_realip_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-mail --with-mail_ssl_module --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-dav-ext-module --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.4.1/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.4.1/debian/modules/ngx_http_substitutions_filter_module

Description

it would be extremely cool if proxy_cache_use_stale had a way to serve stale data to ALL requests including the one that caused the updating.

This would really smooth out all traffic so no client would get unlucky and have to wait for a page load while updating.

Change History (11)

comment:1 Changed 6 years ago by steve.r.weber@…

comment:2 follow-up: Changed 6 years ago by www.google.com/accounts/o8/id?id=AItOawn_EuHb2XimBzYHhglNBas-KnLdMm5omNQ

I agree, this would be a very useful feature to us and other nginx users. I put a bounty of $500 for it at Bountysource https://www.bountysource.com/issues/972735-proxy_cache_use_stale-run-updating-in-new-thread-and-serve-stale-data-to-all

comment:3 in reply to: ↑ 2 Changed 6 years ago by www.google.com/accounts/o8/id?id=AItOawnlcouK0mYE2cIqBm8TG4BHYdnHky1HSFg

Replying to Brian Davis:

I agree, this would be a very useful feature to us and other nginx users. I put a bounty of $500 for it at Bountysource https://www.bountysource.com/issues/972735-proxy_cache_use_stale-run-updating-in-new-thread-and-serve-stale-data-to-all

Well. I had experience with writing Nginx modules as a freelancer. Going to try this one. Brian, when can I pick you on freenode's IRC tomorrow?

I think that changes to current architecture won't be so large (additional flag to enable serving stale data with backend on-line, and revalidating cache as 'updating from stale state'. Probably this additional state (together with 1-2 options) will fit right in, even without introducing dirty hacks with HTTP request being re-issued from server itself.

comment:4 Changed 6 years ago by www.google.com/accounts/o8/id?id=AItOawn_EuHb2XimBzYHhglNBas-KnLdMm5omNQ

Hey Igor, sounds good. I'll make a point to be in the nginx channel on freenode this week between 10am and 5pm CST so we can chat. My nick is bmdavi3. Hopefully a time in there works for you.

comment:5 Changed 5 years ago by steve.r.weber@…

any news on this?

comment:6 Changed 5 years ago by www.google.com/accounts/o8/id?id=AItOawn_EuHb2XimBzYHhglNBas-KnLdMm5omNQ

Not that I know of. I never heard from Igor in IRC so that bounty is still there. However, it sounds like varnish just implemented this feature today with their 4.0.0 release. So if you already have varnish in your stack, it should be easy. We do not currently use Varnish where I work, but this feature is so nice, we are talking about adding it to our stack, just for this.

https://www.varnish-cache.org/lists/pipermail/varnish-announce/2014-April/000696.html

* Background (re)fetch of expired objects. On a cache miss where a stale copy
is available, serve the client the stale copy while fetching an updated copy
from the backend in the background.

comment:7 follow-up: Changed 5 years ago by skylazart@…

Hi, I would like to collaborate working on this new feature.
Brian, do you have any news about it? Is it already done?
Thanks

comment:8 in reply to: ↑ 7 Changed 5 years ago by www.google.com/accounts/o8/id?id=AItOawn_EuHb2XimBzYHhglNBas-KnLdMm5omNQ

Replying to Felipe Cerqueira skylazart:

Hi, I would like to collaborate working on this new feature.
Brian, do you have any news about it? Is it already done?
Thanks

I have no news on this. As far as I know nobody is working on it, so feel free to take a shot at it

comment:9 Changed 4 years ago by richard-fairthorne@…

Last response was 22 months ago. I am interested in looking at this. Is anyone else currently working on it?

comment:10 Changed 7 months ago by snizovtsev@…

Isn't proxy_cache_background_update directive introduced by nginx 1.11.10 (14 Feb 2017) already implements this feature?

comment:11 Changed 7 months ago by mdounin

  • Resolution set to fixed
  • Status changed from new to closed

Yes, thanks.

Note: See TracTickets for help on using tickets.