Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#1636 closed defect (invalid)

CPU 100% with njs on simple base64 decode (2.3.1)

Reported by: 131@… Owned by:
Priority: minor Milestone:
Component: nginx-module Version: 1.14.x
Keywords: njs, performance Cc:
uname -a: Linux 3.16.0-6-amd64 #1 SMP Debian 3.16.57-2 (2018-07-14) x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.14.0
built by gcc 4.9.2 (Debian 4.9.2-10+deb8u1)
built with OpenSSL 1.0.1t 3 May 2016
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

Description

I'm using a simple js_set to base64 decode (via String.bytesFrom)
as proxy_pass destination, and CPU goes too high (100%) under heavy load.

I can reproduce this using ab -c 200 -n 10000

ab -c 200 -n 10000 "http://127.0.0.1/ping"
=> 0% CPU, all good

ab -c 200 -n 10000 "http://127.0.0.1/proxy/string
=> 35% CPU, all good

ab -c 200 -n 10000 "http://127.0.0.1/proxy/js_set
=> 100%CPU, no good.

I'm guessing the resolver also use CPU cycle, but i'm unhappy with current performances, is there something i can tweak ?

(switching to worker_processes auto helped a bit, but 'im still looking for something faster)

Thank you for your kind help

js_include "./conf.d/njs/x_proxypass.js";
js_set $x_proxypass x_proxypass;


server {
    listen       80 default_server;

    gzip off;

    location /ping {
      return 200 "pong";
    }

    location /proxy/string {
      aio threads;
      proxy_buffering off;
      proxy_pass 'https://131.github.io/ping';

    }

    location /proxy/js_set {
      aio threads;
      proxy_buffering off;
      proxy_pass $x_proxypass;
    }
   access_log  /dev/null;
}



function x_proxypass(r) {
  var src = 'aHR0cHM6Ly8xMzEuZ2l0aHViLmlvL3Bpbmc=';
  return String.bytesFrom(src, 'base64');
}

Change History (3)

comment:1 by 131@…, 2 years ago

root@rproxy2:~# apt-cache policy nginx-module-njs
nginx-module-njs:
  Installed: 1.14.0.0.2.3-1~jessie

comment:2 by Maxim Dounin, 2 years ago

Resolution: invalid
Status: newclosed

You are using variables in the proxy_pass directive, and this means that:

  • nginx will use dynamic resolution of the name provided (and this will require resolver to be defined or appropriate upstream block);
  • as long as there is no corresponding upstream block defined, nginx will not be able to cache SSL sessions.

Since you are using https, this implies high CPU usage on uncached SSL handshakes compared to the variant with without variables. Adding a dummy upstream block should resolve this, try something like this:

upstream 131.github.io {
    server 131.github.io:443;
}

Alternatively, you can test njs base64() speed without using proxy_pass:

location /js_set {
    return 200 $x_proxypass;                                                                                             
} 

This should clearly show that it's not njs base64 decode which is the problem in your test.

comment:3 by 131@…, 2 years ago

Switching to a non https url is a full win, ab -n 10000 -c 200 stays under 10% CPU.

Thanks you a lot for your guidance.
And thanks again for njs, a perfect tool that came to me at the right time.

Note: See TracTickets for help on using tickets.