id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc	uname	nginx_version
2389	"""proxy_cache_background_update on"" ignored using subrequest (more exactly: nested subrequest)"	dfex55@…		"The ticket was originally opened for nginx/njs, but it is a problem with the Nginx subrequest mechanism.
See: https://github.com/nginx/njs/issues/573


=== Text from the original issue:

Requesting a location through njs subrequest that has caching enabled with proxy_cache_background_update on and proxy_cache_use_stale updating will never get stale data.

That seems to be a bug, because requesting the same location directly, the cache returns stale data.

In the provided example there are three locations:

{{{/longRunningRequest/}}}:
Has a js handler delaying the result for two seconds

{{{/longRunningRequestCached/}}}:
Has a cache that is valid for one second, allowing stale data while updating and proxy passes to /longRunningRequest/

{{{/longRunningRequestThroughNjs/}}}:
Has a js handler making a subrequest to /longRunningRequestCached/

Expected behavior:
{{{/longRunningRequestCached/}}} and {{{/longRunningRequestThroughNjs/}}} will always return immediately (except for the first request) and serving stale data if needed.

What actually happens:
{{{/longRunningRequestCached/}}} works as described above
{{{/longRunningRequestThroughNjs/}}} is blocking every time the cache entry has to be renewed.





=== Issue explanation by Dmitry Volyntsev:

>What you see, is the limitation of an nginx subrequest mechanism. r.subrequest() creates an internal nginx subrequest with NGX_HTTP_SUBREQUEST_BACKGROUND and NGX_HTTP_SUBREQUEST_IN_MEMORY flags.
>
>proxy_cache_background_update directive is also relies on subrequests. The main problem is: nginx does not support nested subrequest at this point.
>
>Spefically the code of the cache update below checks the r->background flag, which is 1 for r.subrequest() and does not try to make a background update


=== example:

nginx config:
{{{
    js_import myjs.js;

    server {

        location = /longRunningRequestThroughNjs/ {
                js_content myjs.requestToCachedLongRunningRequest;
        }

        location = /longRunningRequest/ {
                js_content myjs.simulateLongRunningRequest;
        }

        location = /longRunningRequestCached/ {
                proxy_connect_timeout 1s;
                proxy_read_timeout 12s;
                proxy_set_header Content-Length """";
                proxy_pass http://127.0.0.1/longRunningRequest/;
                proxy_ignore_headers X-Accel-Expires Expires Cache-Control Set-Cookie;
                proxy_hide_header Set-Cookie;
                proxy_buffering on;

                proxy_cache_use_stale http_404 error timeout invalid_header updating
                                        http_500 http_502 http_503 http_504;

                proxy_cache_background_update on;
                proxy_cache_lock on;
                proxy_cache_lock_age 50s;
                proxy_cache_lock_timeout 60s;
                proxy_cache_revalidate on;
                proxy_cache MYCACHE;
                proxy_cache_valid 200 1s;
                proxy_cache_key $http_accept_encoding$uri;

                add_header x-cache-status $upstream_cache_status;
        }
    }
}}}

myjs.js:
{{{
function delayed(r) {
  r.return(200);
}

function simulateLongRunningRequest(r) {
  setTimeout(delayed, 2000, r);
}

function requestToCachedLongRunningRequest(r) {
  r.subrequest('/longRunningRequestCached/')
    .then(reply => {
      r.return(200);
    })
    .catch(_ => r.return(401));
}

export default { simulateLongRunningRequest, requestToCachedLongRunningRequest };
}}}


"	defect	reopened	major		nginx-core	1.21.x		subrequest background cache		Linux 01b021c85942 5.10.76-linuxkit #1 SMP Mon Nov 8 10:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux	"nginx version: nginx/1.21.6
built by gcc 9.3.0 (Ubuntu 9.3.0-10ubuntu2) 
built with OpenSSL 1.1.1f  31 Mar 2020
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.21.6/debian/debuild-base/nginx-1.21.6=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'"