Opened 8 years ago
Last modified 7 years ago
#1151 new enhancement
Use sched_getaffinity() and CPU_COUNT() for ngx_ncpu on Linux
| Reported by: | Owned by: | ||
|---|---|---|---|
| Priority: | minor | Milestone: | |
| Component: | nginx-core | Version: | 1.11.x |
| Keywords: | Cc: | ||
| uname -a: | Linux e371d562776f 4.8.0-30-generic #32-Ubuntu SMP Fri Dec 2 03:43:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | ||
| nginx -V: |
nginx version: nginx/1.11.6
built by gcc 6.2.0 20161005 (Ubuntu 6.2.0-5ubuntu12) built with OpenSSL 1.1.0c 10 Nov 2016 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -fdebug-prefix-map=/usr/src/nginx-1.11.6=. -fstack-protector-strong -Wformat -Werror=format-security -march=haswell -mtune=haswell -O3 -malign-data=abi' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' |
||
Description
For better integration with systemd cgroups and containers like docker I humbly suggest using an approach similar to the one employed by nproc from coreutils to determine the number of cpus.
Currently, when using a systemd unit (or docker) and limiting nginx to a few cores, setting worker_processes would result in nginx spawning one cpu per core in the system. I suggest to chagne this to use only the amount of CPUs actually available.
Example:
nginx.conf
worker_processes auto;
/etc/systemd/system/nginx.service.d/cpu-affinity.conf
[Service] CPUAffinity=0,1
Should only spawn two processes instead of four. The same goes when using docker, e.g.:
docker run --cpuset-cpus 0,1 --rm -t -i -v /etc/nginx/nginx.conf:/etc/nginx/nginx.conf nginx
should spawn 4 workers (not that the default image will only ever start a single worker).
I wrote a small patch that accomplishes this, not being a C coder or familiar with the nginx code base it's very hacky. It does work however.
Attachments (1)
Change History (3)
by , 8 years ago
| Attachment: | nginx-cgroup-cpu-count.patch added |
|---|
We've considered something like this previously, though rejected. The problem is that it looks impossible to distinguish between CPUs not set in the mask returned by
sched_getaffinity()because they are not available in a particular container, vs. CPUs not set due to previoussched_setaffinity()in the parent process.