Cores on nginx-quiche while running curl command

[Karthikdasari0423@…]

Hi ,

I tried to build nginx with quiche as explained below and tried to run curl with http3 but noticing below cores
​https://github.com/cloudflare/quiche/tree/master/nginx

nginx -V
root@ubuntu:/tmp/cores# /src/nginx-1.16.1/objs/nginx -V
nginx version: nginx/1.16.1 (quiche-ab2b44fa)
built by gcc 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)
built with OpenSSL 1.1.1 (compatible; BoringSSL) (running with BoringSSL)
TLS SNI support enabled
configure arguments: --prefix=/src/nginx-1.16.1 --build=quiche-ab2b44fa --with-http_ssl_module --with-http_v2_module --with-http_v3_module --with-openssl=../quiche/quiche/deps/boringssl --with-quiche=../quiche

curl command and version
root@ubuntu:/tmp/cores# curl -V
curl 8.3.0-DEV (x86_64-pc-linux-gnu) libcurl/8.3.0-DEV OpenSSL/1.1.1s zlib/1.2.11 brotli/1.0.9 msh3/0.6.0.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli HSTS HTTP3 HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL threadsafe UnixSockets

root@ubuntu:/tmp/cores# curl -k -v --http3-only -# -o /tmp/index.html ​https://127.0.0.1:5443/index.html
below is the nginx bt

(gdb) bt
#0 0x000055d6712442c2 in ngx_quic_write_handler (wev=<optimized out>) at src/event/ngx_event_quic.c:411
#1 0x000055d67123456f in ngx_event_process_posted (cycle=cycle@entry=0x55d6737d0700, posted=0x55d6717a0280 <ngx_posted_events>)

at src/event/ngx_event_posted.c:34

#2 0x000055d67123405c in ngx_process_events_and_timers (cycle=cycle@entry=0x55d6737d0700) at src/event/ngx_event.c:267
#3 0x000055d67123bf51 in ngx_worker_process_cycle (cycle=0x55d6737d0700, data=<optimized out>) at src/os/unix/ngx_process_cycle.c:750
#4 0x000055d67123a431 in ngx_spawn_process (cycle=cycle@entry=0x55d6737d0700,

proc=proc@entry=0x55d67123be67 <ngx_worker_process_cycle>, data=data@entry=0x4, name=name@entry=0x55d6715d8236 "worker process",
respawn=respawn@entry=-3) at src/os/unix/ngx_process.c:199

#5 0x000055d67123b537 in ngx_start_worker_processes (cycle=cycle@entry=0x55d6737d0700, n=8, type=type@entry=-3)

at src/os/unix/ngx_process_cycle.c:359

#6 0x000055d67123c62d in ngx_master_process_cycle (cycle=cycle@entry=0x55d6737d0700) at src/os/unix/ngx_process_cycle.c:131
#7 0x000055d671214e0f in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:382
(gdb) quit

nginx error log file

2023/09/06 11:26:06 [alert] 240245#0: worker process 240272 exited on signal 11 (core dumped)
2023/09/06 11:26:08 [alert] 240245#0: worker process 240274 exited on signal 11 (core dumped)
2023/09/06 11:26:12 [alert] 240245#0: worker process 240276 exited on signal 11 (core dumped)
2023/09/06 11:26:26 [alert] 240245#0: worker process 240247 exited on signal 11 (core dumped)
2023/09/06 11:26:27 [alert] 240245#0: worker process 240303 exited on signal 11 (core dumped)
2023/09/06 11:26:29 [alert] 240245#0: worker process 240305 exited on signal 11 (core dumped)
2023/09/06 11:26:33 [alert] 240245#0: worker process 240307 exited on signal 11 (core dumped)
2023/09/06 11:31:05 [alert] 240245#0: worker process 240250 exited on signal 11 (core dumped)
2023/09/06 11:31:06 [alert] 240245#0: worker process 241691 exited on signal 11 (core dumped)
2023/09/06 11:31:08 [alert] 240245#0: worker process 241692 exited on signal 11 (core dumped)
2023/09/06 11:31:12 [alert] 240245#0: worker process 241693 exited on signal 11 (core dumped)
2023/09/06 11:38:52 [alert] 240245#0: worker process 241694 exited on signal 11 (core dumped)
2023/09/06 11:38:53 [alert] 240245#0: worker process 241734 exited on signal 11 (core dumped)
2023/09/06 11:38:55 [alert] 240245#0: worker process 241735 exited on signal 11 (core dumped)
2023/09/06 11:38:59 [alert] 240245#0: worker process 241736 exited on signal 11 (core dumped)
root@ubuntu:/tmp/cores#

root@ubuntu:/tmp/cores# ls -lrth
total 89M
-rwxrwxrwx 1 root root 12M Sep 6 11:31 core.nginx.240250
-rwxrwxrwx 1 root root 12M Sep 6 11:31 core.nginx.241691
-rwxrwxrwx 1 root root 12M Sep 6 11:31 core.nginx.241692
-rwxrwxrwx 1 root root 12M Sep 6 11:31 core.nginx.241693
-rwxrwxrwx 1 root root 12M Sep 6 11:38 core.nginx.241694
-rwxrwxrwx 1 root root 12M Sep 6 11:38 core.nginx.241734
-rwxrwxrwx 1 root root 12M Sep 6 11:38 core.nginx.241735
-rwxrwxrwx 1 root root 12M Sep 6 11:38 core.nginx.241736
root@ubuntu:/tmp/cores#

am i missing anything here?

[Karthikdasari0423@…]

Bt for all cores are same
`
(gdb) core-file core.nginx.240250
warning: Can't open file /dev/zero (deleted) during file-backed mapping note processing
[New LWP 240250]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `nginx: worker process '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000055d6712442c2 in ngx_quic_write_handler (wev=<optimized out>) at src/event/ngx_event_quic.c:411
411 ngx_post_event(wev, &ls->quic_blocked_events);
(gdb) bt
#0 0x000055d6712442c2 in ngx_quic_write_handler (wev=<optimized out>) at src/event/ngx_event_quic.c:411
#1 0x000055d67123456f in ngx_event_process_posted (cycle=cycle@entry=0x55d6737d0700, posted=0x55d6717a0280 <ngx_posted_events>)

at src/event/ngx_event_posted.c:34

#2 0x000055d67123405c in ngx_process_events_and_timers (cycle=cycle@entry=0x55d6737d0700) at src/event/ngx_event.c:267
#3 0x000055d67123bf51 in ngx_worker_process_cycle (cycle=0x55d6737d0700, data=<optimized out>) at src/os/unix/ngx_process_cycle.c:750
#4 0x000055d67123a431 in ngx_spawn_process (cycle=cycle@entry=0x55d6737d0700,

proc=proc@entry=0x55d67123be67 <ngx_worker_process_cycle>, data=data@entry=0x4, name=name@entry=0x55d6715d8236 "worker process",
respawn=respawn@entry=-3) at src/os/unix/ngx_process.c:199

#5 0x000055d67123b537 in ngx_start_worker_processes (cycle=cycle@entry=0x55d6737d0700, n=8, type=type@entry=-3)

at src/os/unix/ngx_process_cycle.c:359

#6 0x000055d67123c62d in ngx_master_process_cycle (cycle=cycle@entry=0x55d6737d0700) at src/os/unix/ngx_process_cycle.c:131
#7 0x000055d671214e0f in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:382
`
`
(gdb) core-file core.nginx.241691
warning: Can't open file /dev/zero (deleted) during file-backed mapping note processing
[New LWP 241691]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `nginx: worker process '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000055d6712442c2 in ngx_quic_write_handler (wev=<optimized out>) at src/event/ngx_event_quic.c:411
411 ngx_post_event(wev, &ls->quic_blocked_events);
(gdb) bt
#0 0x000055d6712442c2 in ngx_quic_write_handler (wev=<optimized out>) at src/event/ngx_event_quic.c:411
#1 0x000055d67123456f in ngx_event_process_posted (cycle=cycle@entry=0x55d6737d0700, posted=0x55d6717a0280 <ngx_posted_events>)

at src/event/ngx_event_posted.c:34

#2 0x000055d67123405c in ngx_process_events_and_timers (cycle=cycle@entry=0x55d6737d0700) at src/event/ngx_event.c:267
#3 0x000055d67123bf51 in ngx_worker_process_cycle (cycle=0x55d6737d0700, data=<optimized out>) at src/os/unix/ngx_process_cycle.c:750
#4 0x000055d67123a431 in ngx_spawn_process (cycle=cycle@entry=0x55d6737d0700, proc=0x55d67123be67 <ngx_worker_process_cycle>,

data=0x4, name=0x55d6715d8236 "worker process", respawn=respawn@entry=4) at src/os/unix/ngx_process.c:199

#5 0x000055d67123c8c8 in ngx_reap_children (cycle=0x55d6737d0700) at src/os/unix/ngx_process_cycle.c:622
#6 ngx_master_process_cycle (cycle=cycle@entry=0x55d6737d0700) at src/os/unix/ngx_process_cycle.c:175
#7 0x000055d671214e0f in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:382
`

[Maxim Dounin]

It looks like you are having issues nginx with 3rd party modifications. Unfortunately, we can't help here. Consider using unmodified nginx as available on ​nginx.org instead.

[Karthikdasari0423@…]

Just FYI,i am not using any modified nginx.nginx i am using is unmodified only

Below is the nginx i am using

​http://nginx.org/download/nginx-1.16.1.tar.gz

followed below steps to build nginx,
​https://github.com/cloudflare/quiche/tree/master/nginx

May i know what made you to think i am using modified nginx instead of original nginx.

[Maxim Dounin]

The "steps to build" you've referenced contain a 140 KB patch file with modifications to nginx source code. Further, these modifications are clearly visible in the backtraces you've provided - which also suggests that these 3rd party modifications are the immediate reason for the segmentation faults you are seeing.

Unfortunately, we are not able to help with 3rd party modifications. As previously suggested, consider using unmodified nginx instead, as available from ​nginx.org. In particular, recent mainline versions contain ​expiremental HTTP/3 support.

[Karthikdasari0423@…]

okay,got it

i already built nginx with 1.25.2 version and i should really appreciate nginx for that.

I thought of building nginx with quiche also,so tried above