limit_req_zone is incorrect block requests

[Maxim]

Hello!

I try setup limit_req_zone for load balancing.

My config:

limit_req_zone $binary_remote_addr zone=block_1:10m rate=50r/s;

server {
        server_name   test.local;
        listen        80;

        location      / {
           limit_req zone=block_1;
           limit_req_status 418;

           root /etc/nginx/conf.d/;
        }

    }

For tests i use yandex-tank with config:

phantom:
  address: mysite.com:80  # [Target's address]:[target's port]
  uris:
    - /
  load_profile:
    load_type: rps # schedule load by defining requests per second
    schedule: line(50, 50, 1m) # starting from 1rps growing linearly to 10rps during 10 minutes
  headers:
    - "[Referer: /]"
  ssl: false
console:
  enabled: false # enable console output
telegraf:
  enabled: false # let's disable telegraf monitoring for the first time
overload:
  enabled: true
  package: yandextank.plugins.DataUploader
  token_file: "token.txt"

and results is very strange:
for 30rps in fact ~18rps
​https://overload.yandex.net/520419#tab=test_data&tags=&plot_groups=main&machines=&metrics=&slider_start=1651236460&slider_end=1651236520

for 50rps in fact ~30rps
​https://overload.yandex.net/520421#tab=test_data&tags=&plot_groups=main&machines=&metrics=&slider_start=1651236627&slider_end=1651236687

docker-compose.yml

version: '3.5'

services:
    nginx:
      image: nginx
      volumes:
        - ./conf:/etc/nginx/conf.d
      ports:
        - '7777:80'

[Maxim Dounin]

You are using ​limit_req without any burst set, so every request which comes before exactly 1/30 of a second passes will be rejected. Test results suggest this sometimes happens, so the number of the requests actually allowed is less than the theoretical maximum allowed. With a client which maintains internals between requests more precisely you should be able to reach limits set more closely.

On the other hand, it is quite normal that intervals between requests vary in the real life. To work in such real-life conditions, nginx provides the burst parameter, which makes it possible to tolerate traffic spikes while still maintaining the limit in the long run. Usually it is a good idea to configure burst to a number of requests allowed per couple of seconds. That is, something like:

limit_req zone=block_1 burst=100;

should be good enough for rate=50r/s. Depending on the particular use case, nodelay might be also a good idea.

Note well that 50r/s is a very large rate when limiting individual clients, and with burst properly set you may want to use much smaller values.

Hope this helps. If you need further help with configuring nginx, please use ​support options available.