#1403 closed defect (wontfix)
nginx module cannot use libpcre
| Reported by: | Owned by: | ||
|---|---|---|---|
| Priority: | minor | Milestone: | |
| Component: | nginx-core | Version: | 1.10.x |
| Keywords: | Cc: | ||
| uname -a: | Linux <somename> 4.9.45-yocto-custom #3 SMP Wed Sep 27 14:10:54 CEST 2017 i686 i686 i386 GNU/Linux | ||
| nginx -V: | configure arguments: --with-ipv6 --with-http_ssl_module --with-http_auth_request_module --with-http_gzip_static_module --with-http_v2_module --with-cc='i586-poky-linux-gcc -Wl,--hash-style=gnu -DNGX_ENABLE_SYSLOG' --with-pcre --sbin-path=/usr/sbin/nginx --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --error-log-path=/var/log/nginx_error --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx_access --http-client-body-temp-path=/var/lib/nginx/client_body_temp --http-proxy-temp-path=/var/lib/nginx/proxy_temp --http-fastcgi-temp-path=/var/lib/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/lib/nginx/uwsgi_temp --http-scgi-temp-path=/var/lib/nginx/scgi_temp --add-module=../rrdgraph --add-module=../echomod --with-cc='i586-poky-linux-gcc -m32 -march=i586 --sysroot=<some-sysroot> -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed' | ||
Description
Using the rrdgraph module https://github.com/evanmiller/mod_rrd_graph, and with glib having switched to using the libpcre system library, I found that libpcre becomes unusable for that module (and theoretically for all other modules, too), because src/core/ngx_regex sets pcre_malloc = ngx_regex_malloc. This, of course works for ngx_regex_compile, which initializes the pool used by ngx_regex_malloc and clears it after regex compilation.
libpcre error message:
cannot compile regular expression: Error while compiling regular expression ^(?:[^%]+|%%)*%[-+ 0#]?[0-9]*(?:[.][0-9]+)?l[eEfFgG](?:[^%]+|%%)*(?:%s)?(?:[^%]+|%%)*$ at char 86: failed to get memory (^(?:[^%]+|%%)*%[-+ 0#]?[0-9]*(?:[.][0-9]+)?l[eEfFgG](?:[^%]+|%%)*(?:%s)?(?:[^%]+|%%)*$)
For an nginx module (in our case compiled statically into nginx), this means, however, that using libpcre regex compilation is no longer possible, because ngx_regex_malloc does not have any buffer available when called from somewhere else.
What first came to my mind was to just exchange the allocation mechanism before calling the rrdlibrary and change it back afterwards, but I'm quite sure this would get me into serious multithreading trouble.
Globally changing the pcre_malloc to standard malloc is probably even worse due to the allocated memory not being cleaned up, because this is usually done by the pools (right?) and also probably decreasing performance.
Any ideas on how to solve this without doing any damage?
A working alternative is to use the glib's embedded libpcre, but this is quite outdated and judging from https://bugzilla.gnome.org/show_bug.cgi?id=740573 will be removed soon.
In general nginx assumes that any regex compilation should be done via
ngx_regex_compile()which takes care of setting appropriate global variables, notably pool to allocate memory from. Once finished, it is intentionally doesngx_pcre_pool = NULLto ensure no additional allocations will be done. This expectedly causes problems when PCRE functions are used directly.Obviously enough, in the case of the particular module it is not possible to switch to using nginx infrastructure (which would be best possible solution), as PCRE is used deeply in the rrd library. A possible solution would be to set
pcre_malloc/pcre_freebefore the library call, and restore them afterwards. This should work at least as long as the library does not try to create threads and do some background work (and certainly there will be other problems if a library will try to create threads).I doubt there is a better solution since memory allocation functions in PCRE are global. PCRE2 might be better here as it allows to specify private allocation functions via
pcre2_general_context_create()instead of via global variables, though it's yet to be adopted.