Inconsistency between url encoding/decoder when using proxy_cache_background_update
|Reported by:||Amotz Terem||Owned by:|
|uname -a:||Linux my.server 4.4.0-34-generic #53-Ubuntu SMP Wed Jul 27 16:06:39 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux|
nginx version: nginx/1.13.6
built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4)
configure arguments: --with-debug
It seems that there is some inconsistency with proxy_cache_background_update in regard to encoded "/". Which results in first request (MISS) to pass through encoded (not parsed) to the remote server. But later after expiration of the cache document, while ngnix is doing proxy_cache_background_update this time the remote server will get the request decoded (i.e different from the original request when it had a cache MISS ). This will results in cache never refreshing and STALE will be returned until the cache is cleared
Use "proxy_cache_background_update on" in nginx
In remove sever set headers with max-age=30,stale-while-revalidate=300
make a request with character "%2F" (curl localhost:8000/myserver/at%2Faa) - this call will be cache-status MISS and will pass to server.
make the same call again - cache-status should be HIT
Wait for 30 seconds (same as max-age) and make the call again.
The exact same call (as original) should go to remote server.
cache status should return STALE
on following call cache status should be HIT
The call in remote server is different from the original call (this time the %2F is parsed to / on the remote server)
cache status return STALE -same as before
on following call cache status continue to return STALE
I'm not going into the discussion of whether nginx should or should not decode the url when proxying. I'm saying that the call of proxy_cache_background_update should be 100% identical to the original call that went through the first time ( when it was a cache MISS).
I'm sure that there are many ways to workaround if I want to. But I'm interested if you would consider this a real bug because if so I'm interested to try and fix it.
Looking at the log I can see that in the first try it is using "http cache key" the is not decoded. But in the second try (when returning from call) it is using "http cache key" that is decoding, this demonstrate the issue from ngnix POV. (from remote server it is different request path).
I tried playing with the function ngx_http_upstream_cache_background_update and so the final call will be as original, but didn't quite manage to fix it.
Please share you opinion