Nginx starting more worker processes when modsec rules are enabled

[marferiky@…]

Hi there, we run Nginx in a Docker container as a sidecar in a pod in a Kubernetes cluster.

We have the ModSec module (version 3.0.3) enabled for Nginx.

In our Nginx configuration, we set the number of worker processes to be 16 (we have 16 cpu cores on our machines):

    worker_processes 16;
    events {
      worker_connections 1024;
    }

Without modsec enabled, it all works fine, we can see in the access logs that requests are processed fine, there are 16 worker processes running and the memory usage is not higher than 200-300MB.

However, when the modsec is enabled, the initial number of worker processes in nginx is 16, but over a period of 12 hours, the number of worker processes increases whenever a given modsec rule is matched by an HTTP request (in our case it is ruleID: 920350).

The number of worker processes increases until the memory limit (2GB, set at a kubernetes level) is met (at that point, nginx started up to 500 worker processes!)

When it hits the memory limit, nginx stops handling requests (according to the access log, we cannot see any requests being handled anymore). Furthermore, the issue is causing the entire machine where the nginx container is running to crash.

I am wondering why is nginx starting more worker processes than the configured number?

[maxim]

Hi,

can you check whether nginx demonstrates the same behaviour with modsecurity disabled?

I see nothing in OWASP ruleid 920350 here ​https://github.com/SpiderLabs/owasp-modsecurity-crs/ that can trigger such behaviour unless some subtle bug in the modsecurity module.

Thanks,

Maxim

[marferiky@…]

nginx worker processes

[marferiky@…]

hi Maxim, thanks for getting back!

When modsecurity is disabled, it all works fine: the number of worker processes in nginx is 16 as configured and the whole container memory usage is below 300MB.

The behavior is only happening when modsecurity is enabled and that rule matches HTTP requests

[maxim]

I would update both nginx and, more important, libmodsecurity and nginx modsecurity module to the most recent snapshot from their v3/master branch and try to reproduce this issue again.

So far it sounds like a bug in the modsecurity and should be reported the modsecurity developers ​https://github.com/SpiderLabs/ModSecurity

Thanks,

Maxim

[marferiky@…]

ok, I'll try that, thanks!

[Maxim Dounin]

Feedback timeout. Likely a bug in modsecurity.