id,summary,reporter,owner,description,type,status,priority,milestone,component,version,resolution,keywords,cc,uname,nginx_version 1030,100% CPU with hash consistent balancing method without live upstreams,Alex,,"Good day, For caching purposes I have the following upstream configuration: upstream upstream1 { hash $arg_1$arg_2$arg_3 consistent; server backend1:80 weight=100; server backend2:80 weight=100; server backend3:80 weight=100; server backend4:80 weight=100; zone upstream 10m; keepalive 512; } Once backend servers experience problems, and the proxy considers all of it as unavailable (leading to ""no live upstreams while connecting to upstream"" in error logs) then all workers go 100% CPU. Single restarts usually don't help. It ends with either few nginx restarts or wait for nginx to recover (yes, it recovers on its own after a while) or by taking care of backend. I guess all 3 lead to same thing - backend availability. Since backup upstream can't be used with 'hash' balancing algo, I tried adding another upstream (leading to nginx itself and some special response) with very low weight: server backend1:80 weight=1000; ... server localhost:81 weight=1; But even that didn't help. That ""backup"" location stops receiving requests. And once all other upstreams are down it also goes down. So CPU goes 100%, and response time grows to unreal values like 1-2+ minutes regardless the timeouts set (well, server's CPU is 100% in use...) Without 'consistent' it works fine as well as with basic RR balancing method that been in use so far. Also, that proxy works with 10k-60k req/sec, and the problem might happen at any time. Thanks! ",defect,closed,major,,nginx-core,1.11.x,fixed,,,Linux RP1 3.10.0-327.18.2.el7.x86_64 #1 SMP Thu May 12 11:03:55 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux,"nginx version: nginx/1.11.1 (TFO custom build) built by gcc 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC) built with OpenSSL 1.0.1e-fips 11 Feb 2013 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error_log --http-log-path=/var/log/nginx --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --user=nginx --group=nginx --build='TFO custom build' --with-http_stub_status_module --with-http_ssl_module --with-http_realip_module --without-http_ssi_module --without-http_userid_module --without-http_geo_module --without-http_referer_module --without-http_uwsgi_module --without-http_scgi_module --without-http_memcached_module --without-http_browser_module --without-http_upstream_ip_hash_module --without-http_upstream_least_conn_module --with-http_gunzip_module --add-module=./ngx_devel_kit-0.3.0/ --add-module=./lua-nginx-module-0.10.5 --add-module=./headers-more-nginx-module-0.30 --with-pcre-jit --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -DTCP_FASTOPEN=23' "