Opened 5 years ago

Last modified 4 years ago

#1719 new enhancement

Enhance proxy_cache_min_uses directive

Reported by: vadim.lazovskiy@… Owned by:
Priority: minor Milestone:
Component: other Version: 1.15.x
Keywords: Cc:
uname -a:
nginx -V: nginx version: nginx/1.15.8
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
built with OpenSSL 1.1.0j 20 Nov 2018
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.15.8/debian/debuild-base/nginx-1.15.8=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

Description

When using nginx for caching large high traffic media it may have sense to use proxy_cache_min_uses directive. But currently it accepts only numeric constant, so it's functionality is limited.
There are valid use cases requiring some smart values for proxy_cache_min_uses.

Let's assume we have a nginx proxy with caching on SSD. These kind of drives tend to wear out quickly when used in high-write applications. To save IO it might be beneficial to cache only realy frequent requests and keep cache as hot as possible.

From my point of view setting this directive automatically to the value above least recent used entry score stored in appropriate cache_zone will do the trick. As the result it will reduce cache writes for infrequent requests and keep it red-hot for it size.

Change History (1)

comment:1 by Maxim Dounin, 4 years ago

Consider the following scenario:

  1. A single resource suddenly becomes very popular and gets 1 million hits ("slashdotted"). It is served from cache and all of these hits are counted in "uses" of the cache entry.
  2. A resource is forgotten and no longer requested, so after some time it becomes least recently used.

If, as suggested, we'll set proxy_cache_min_uses based on the uses counter of this least recently used entry, this will result in dramatic change of the cache behaviour: it will effectively prevent other resources from entering the cache. Further, it might stuck in this state forever (till nginx restart), as no other resources entering the cache might result in proxy_cache_min_uses never updated again.

While auto-tuning proxy_cache_min_uses might be interesting at least in some setups, the particular algorithm seems to be flawed.

Note: See TracTickets for help on using tickets.