Opened 10 months ago

Closed 9 months ago

Last modified 8 months ago

#2605 closed defect (fixed)

NGINX + BoringSSL build error (NGINX 1.25.4 required Openssl)

Reported by: Karthikdasari0423@… Owned by:
Priority: trivial Milestone: nginx-1.26
Component: http/3 Version: 1.25.x
Keywords: Cc:
uname -a: Linux ubuntu 5.15.0-83-generic #92-Ubuntu SMP Mon Aug 14 09:30:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.25.3 (nginx-quic)
built by gcc 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)
built with OpenSSL 1.1.1 (compatible; BoringSSL) (running with BoringSSL)
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_v3_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --build=nginx-quic --with-debug --with-http_v3_module --with-cc-opt=-I/src/boringssl/include --with-ld-opt='-L/src/boringssl/build/ssl -L/src/boringssl/build/crypto'

Description (last modified by Karthikdasari0423@…)

From nginx 1.25.4,nginx is throwing below error but worked fine with nginx 1.25.3.

with nginx 1.25.4:-
==================
command used is :-
auto/configure nginx -V 2>&1 | sed "s/ \-\-/ \\\ \n\t--/g" | grep -v -e 'http-geoip2' | grep "\-\-" | grep -ve opt= -e param= -e build= --build=nginx-quic --with-debug --with-http_v3_module --with-cc-opt="-I/src/boringssl/include" --with-ld-opt="-L/src/boringssl/build/ssl -L/src/boringssl/build/crypto"

checking for getaddrinfo() ... found
checking for PCRE2 library ... not found
checking for PCRE library ... found
checking for PCRE JIT support ... found
checking for OpenSSL library ... not found
checking for OpenSSL library in /usr/local/ ... not found
checking for OpenSSL library in /usr/pkg/ ... not found
checking for OpenSSL library in /opt/local/ ... not found

auto/configure: error: SSL modules require the OpenSSL library.
You can either do not enable the modules, or install the OpenSSL library
into the system, or build the OpenSSL library statically from the source
with nginx by using --with-openssl=<path> option.

with 1.25.3:-
============
command used is :-
auto/configure nginx -V 2>&1 | sed "s/ \-\-/ \\\ \n\t--/g" | grep -v -e 'http-geoip2' | grep "\-\-" | grep -ve opt= -e param= -e build= --build=nginx-quic --with-debug --with-http_v3_module --with-cc-opt="-I/src/boringssl/include" --with-ld-opt="-L/src/boringssl/build/ssl -L/src/boringssl/build/crypto"

checking for PCRE2 library ... not found
checking for PCRE library ... found
checking for PCRE JIT support ... found
checking for OpenSSL library ... found
checking for OpenSSL QUIC support ... found
checking for zlib library ... found
creating objs/Makefile

Configuration summary

+ using threads
+ using system PCRE library
+ using system OpenSSL library
+ using system zlib library

may i know why nginx 1.25.3 worked with system Openssl library and not nginx 1.25.4?

Change History (16)

comment:1 by Karthikdasari0423@…, 10 months ago

i have installed openssl in my system

root@ubuntu:/src/nginx-quic# openssl version
OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
root@ubuntu:/src/nginx-quic#

comment:2 by Karthikdasari0423@…, 10 months ago

am i missing anything here

Last edited 10 months ago by Karthikdasari0423@… (previous) (diff)

comment:3 by Karthikdasari0423@…, 10 months ago

Description: modified (diff)

comment:4 by Karthikdasari0423@…, 10 months ago

root@ubuntu:/src/nginx-quic# uname -a
Linux ubuntu 5.15.0-83-generic #92-Ubuntu SMP Mon Aug 14 09:30:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
root@ubuntu:/src/nginx-quic# nginx -V
nginx version: nginx/1.25.4
built by gcc 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)
built with OpenSSL 3.0.2 15 Mar 2022
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_v3_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -ffile-prefix-map=/data/builder/debuild/nginx-1.25.4/debian/debuild-base/nginx-1.25.4=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'
root@ubuntu:/src/nginx-quic#

comment:5 by Karthikdasari0423@…, 10 months ago

installation fine if i remove
--with-cc-opt="-I/src/boringssl/include" --with-ld-opt="-L/src/boringssl/build/ssl -L/src/boringssl/build/crypto"

in reply to:  5 comment:6 by rttwyjz@…, 10 months ago

Replying to Karthikdasari0423@…:

installation fine if i remove
--with-cc-opt="-I/src/boringssl/include" --with-ld-opt="-L/src/boringssl/build/ssl -L/src/boringssl/build/crypto"

Hi, I also encountered the same problem. For details, see https://trac.nginx.org/nginx/ticket/2606. The temporary solution is to switch to
https://github.com/google/boringssl
/commit/c39e6cd9ec5acebb6de2adffc03cfe03b07f08ab This commit recompiles boringssl.
The command is as follows:

git clone https://github.com/google/boringssl.git
cd boringssl
git reset --hard c39e6cd9ec5acebb6de2adffc03cfe03b07f08ab
mkdir build
cd build
cmake -GNinja ..
ninja
cd ../..

But I think this is not the final solution, but it can alleviate your problem

comment:7 by Karthikdasari0423@…, 10 months ago

Thanks, will give it a try

comment:8 by Roman Arutyunyan, 9 months ago

Since https://github.com/google/boringssl/commit/c52806157c97105da7fdc2b021d0a0fcd5186bf3 BoringSSL libssl requires a C++ runtime to be linked. The solution is different depending on platform and compiler.

For MacOS with clang it's enough to add -lc++ to the linker options while configuring nginx:

auto/configure --with-cc-opt="-I../boringssl/include"
               --with-ld-opt="-L../boringssl/build/ssl -L../boringssl/build/crypto -lc++"

For Linux with clang the solution is similar:

auto/configure --with-cc=clang 
               --with-cc-opt="-I../boringssl/include"
               --with-ld-opt="-L../boringssl/build/ssl -L../boringssl/build/crypto -lstdc++"

For Linux with gcc you have to use c++ linker instead. Since the same tool is used to compile and link nginx, you also need to add -x c to compiler options for proper C compilation:

auto/configure --with-cc=c++
               --with-cc-opt="-I../boringssl/include -x c"
               --with-ld-opt="-L../boringssl/build/ssl -L../boringssl/build/crypto"

in reply to:  8 comment:9 by iz8mbw@…, 9 months ago

Replying to Roman Arutyunyan:

Since https://github.com/google/boringssl/commit/c52806157c97105da7fdc2b021d0a0fcd5186bf3 BoringSSL libssl requires a C++ runtime to be linked. The solution is different depending on platform and compiler.

For MacOS with clang it's enough to add -lc++ to the linker options while configuring nginx:

auto/configure --with-cc-opt="-I../boringssl/include"
               --with-ld-opt="-L../boringssl/build/ssl -L../boringssl/build/crypto -lc++"

For Linux with clang the solution is similar:

auto/configure --with-cc=clang 
               --with-cc-opt="-I../boringssl/include"
               --with-ld-opt="-L../boringssl/build/ssl -L../boringssl/build/crypto -lstdc++"

For Linux with gcc you have to use c++ linker instead. Since the same tool is used to compile and link nginx, you also need to add -x c to compiler options for proper C compilation:

auto/configure --with-cc=c++
               --with-cc-opt="-I../boringssl/include -x c"
               --with-ld-opt="-L../boringssl/build/ssl -L../boringssl/build/crypto"

I have tried to build on Limux with:

auto/configure --with-cc=c++
                --with-cc-opt="-I../boringssl/include -x c"
                --with-ld-opt="-L../boringssl/build/ssl -L../boringssl/build/crypto"

now it's better but I have errors on PCRE:

c++ -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I/opt/boringsslnew/include -x c -Wno-deprecated-declarations  -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I /opt/pcre2/src/ -I objs -I src/http -I src/http/modules -I src/http/v2 -I src/http/v3 -I src/http/modules/perl -I /root/autobuild/ngx_brotli/deps/brotli/c/include -I src/stream \
        -o objs/addon/ngx_http_substitutions_filter_module-master/ngx_http_subs_filter_module.o \
        /root/autobuild/ngx_http_substitutions_filter_module-master/ngx_http_subs_filter_module.c
c++ -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I/opt/boringsslnew/include -x c -Wno-deprecated-declarations  -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I /opt/pcre2/src/ -I objs -I src/http -I src/http/modules -I src/http/v2 -I src/http/v3 -I src/http/modules/perl -I /root/autobuild/ngx_brotli/deps/brotli/c/include -I src/stream \
        -o objs/addon/filter/ngx_http_brotli_filter_module.o \
        /root/autobuild/ngx_brotli/filter/ngx_http_brotli_filter_module.c
c++ -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I/opt/boringsslnew/include -x c -Wno-deprecated-declarations  -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I /opt/pcre2/src/ -I objs -I src/http -I src/http/modules -I src/http/v2 -I src/http/v3 -I src/http/modules/perl -I /root/autobuild/ngx_brotli/deps/brotli/c/include -I src/stream \
        -o objs/addon/static/ngx_http_brotli_static_module.o \
        /root/autobuild/ngx_brotli/static/ngx_http_brotli_static_module.c
c++ -c -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I/opt/boringsslnew/include -x c -Wno-deprecated-declarations -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I /opt/pcre2/src/ -I objs \
        -o objs/ngx_modules.o \
        objs/ngx_modules.c
cd /opt/pcre2 \
&& make libpcre2-8.la
make[2]: Entering directory '/opt/pcre2'
rm -f src/pcre2_chartables.c
ln -s /opt/pcre2/src/pcre2_chartables.c.dist /opt/pcre2/src/pcre2_chartables.c
  CC       src/libpcre2_8_la-pcre2_auto_possess.lo
  CC       src/libpcre2_8_la-pcre2_chkdint.lo
  CC       src/libpcre2_8_la-pcre2_compile.lo
src/pcre2_compile.c: In function 'pcre2_code_8* pcre2_code_copy_8(const pcre2_code_8*)':
src/pcre2_compile.c:1205:30: error: invalid conversion from 'void*' to 'pcre2_code_8*' {aka 'pcre2_real_code_8*'} [-fpermissive]
 1205 | newcode = code->memctl.malloc(code->blocksize, code->memctl.memory_data);
      |           ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                              |
      |                              void*
src/pcre2_compile.c: In function 'pcre2_code_8* pcre2_code_copy_with_tables_8(const pcre2_code_8*)':
src/pcre2_compile.c:1240:30: error: invalid conversion from 'void*' to 'pcre2_code_8*' {aka 'pcre2_real_code_8*'} [-fpermissive]
 1240 | newcode = code->memctl.malloc(code->blocksize, code->memctl.memory_data);
      |           ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                              |
      |                              void*
src/pcre2_compile.c:1245:32: error: invalid conversion from 'void*' to 'uint8_t*' {aka 'unsigned char*'} [-fpermissive]
 1245 | newtables = code->memctl.malloc(TABLES_LENGTH + sizeof(PCRE2_SIZE),
      |             ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                |
      |                                void*
 1246 |   code->memctl.memory_data);
      |   ~~~~~~~~~~~~~~~~~~~~~~~~~
src/pcre2_compile.c: In function 'int parse_regex(PCRE2_SPTR8, uint32_t, BOOL*, compile_block_8*)':
src/pcre2_compile.c:4899:32: error: invalid conversion from 'void*' to 'named_group_8*' [-fpermissive]
 4899 |           cb->cx->memctl.malloc(newsize * sizeof(named_group),
      |           ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                |
      |                                void*
 4900 |           cb->cx->memctl.memory_data);
      |           ~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/pcre2_compile.c: In function 'pcre2_code_8* pcre2_compile_8(PCRE2_SPTR8, size_t, uint32_t, int*, size_t*, pcre2_compile_context_8*)':
src/pcre2_compile.c:10508:58: error: invalid conversion from 'void*' to 'uint32_t*' {aka 'unsigned int*'} [-fpermissive]
10508 |   uint32_t *heap_parsed_pattern = ccontext->memctl.malloc(
      |                                   ~~~~~~~~~~~~~~~~~~~~~~~^
      |                                                          |
      |                                                          void*
10509 |     (parsed_size_needed + 1) * sizeof(uint32_t), ccontext->memctl.memory_data);
      |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/pcre2_compile.c:10539:43: error: invalid conversion from 'void*' to 'uint32_t*' {aka 'unsigned int*'} [-fpermissive]
10539 |     cb.groupinfo = ccontext->memctl.malloc(
      |                    ~~~~~~~~~~~~~~~~~~~~~~~^
      |                                           |
      |                                           void*
10540 |       (2 * (cb.bracount + 1))*sizeof(uint32_t), ccontext->memctl.memory_data);
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [Makefile:2584: src/libpcre2_8_la-pcre2_compile.lo] Error 1
make[2]: *** Waiting for unfinished jobs....
c++ -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I/opt/boringsslnew/include -x c -Wno-deprecated-declarations -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/lib/x86_64-linux-gnu/perl/5.34/CORE -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I /opt/pcre2/src/ -I objs -I src/http -I src/http/modules -I src/http/v2 -I src/http/v3 -I src/http/modules/perl -I /root/autobuild/ngx_brotli/deps/brotli/c/include \
        -o objs/src/http/modules/perl/ngx_http_perl_module.o \
        src/http/modules/perl/ngx_http_perl_module.c
make[2]: Leaving directory '/opt/pcre2'
make[1]: *** [objs/Makefile:1839: /opt/pcre2/.libs/libpcre2-8.a] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/root/autobuild/nginx-1.25.4'
make: *** [Makefile:10: build] Error 2
make: Leaving directory '/root/autobuild/nginx-1.25.4'

In

/opt/pcre2

I have PCRE version 10.43 extractewd from source code.

comment:10 by Karthikdasari0423@…, 9 months ago

It worked for me with latest boringssl and
auto/configure --with-cc=c++ --with-cc-opt="-I../boringssl/include -x c" --with-ld-opt="-L../boringssl/build/ssl -L../boringssl/build/crypto"

Cause maybe i am not using PCRE2
checking for PCRE2 library ... not found
checking for PCRE library ... found
checking for PCRE JIT support ... found

Last edited 9 months ago by Karthikdasari0423@… (previous) (diff)

comment:11 by Karthikdasari0423@…, 9 months ago

Installed pcre2 with sudo apt install libpcre2-dev
and tried installing nginx
seems to be working fine in my setup

checking for openat(), fstatat() ... found
checking for getaddrinfo() ... found
checking for PCRE2 library ... found
checking for OpenSSL library ... found
checking for OpenSSL QUIC support ... found
checking for zlib library ... found
creating objs/Makefile

Maybe you might be missing something, pls check once
my linux is 22

comment:12 by iz8mbw@…, 9 months ago

Hi all.
I was able to fix.
Since I would like to use the latest version of PCRE (and so not the PCRE version available into the Operating System), I normally configure/compile nginx by specify the source code of PCRE, I do this with

--with-pcre=/opt/pcre2_source_code

.

Since now we are "forcing" the nginx building with

--with-cc=c++

to fix the BoringSSL issues, I suppose this broke the building of PCRE.

So what I did is to build PCRE (before to build nginx) and after, when going to configure nginx, provide to nginx the path where find PCRE.

This is the part of "configure" about BoringSSL (previously built) and PCRE (previously built):

configure --with-cc=c++ --with-cc-opt="-I/opt/boringsslnew/include -I/opt/pcre2built/include -x c" --with-ld-opt="-L/opt/boringsslnew/build/ssl -L/opt/boringsslnew/build/crypto -L/opt/pcre2built/lib"

Best Regards.

comment:13 by Roman Arutyunyan, 9 months ago

Resolution: fixed
Status: newclosed

Thanks for the update. Closing the ticket now.

comment:14 by Roman Arutyunyan, 9 months ago

Summary: NGINX 1.25.4 required OpensslNGINX + BoringSSL build error (NGINX 1.25.4 required Openssl)

comment:15 by iz8mbw@…, 9 months ago

just as note, using the "latest" version of BoringSSL, TLS v1.2 is not more available in nginx and nginx works only with TLS 1.3

comment:16 by m.herasimovich, 8 months ago

Milestone: nginx-1.25nginx-1.26

Milestone renamed

Note: See TracTickets for help on using tickets.