General protection fault on 1.24 and 1.25. 1.22.1 is OK

[Tan Shao Yi]

Hi, I am seeing nginx fail with a general protection fault after a few seconds/minutes of running:

2023-06-12T14:58:05.667643+08:00 hostname kernel: [24106.297832] traps: nginx[95896] general protection fault ip:560e391f0900 sp:7ffc56ece5d0 error:0 in nginx[560e3919f000+ce000]
2023-06-12T14:58:05.731645+08:00 hostname kernel: [24106.360325] traps: nginx[95897] general protection fault ip:560e391f0900 sp:7ffc56ece550 error:0 in nginx[560e3919f000+ce000]
2023-06-12T14:58:05.771640+08:00 hostname kernel: [24106.399470] traps: nginx[95898] general protection fault ip:560e391f0900 sp:7ffc56ece550 error:0 in nginx[560e3919f000+ce000]
2023-06-12T14:58:05.808863+08:00 hostname kernel: [24106.436569] traps: nginx[95899] general protection fault ip:560e391f0900 sp:7ffc56ece550 error:0 in nginx[560e3919f000+ce000]

The strange thing is this seems to happen in 1.24 and 1.25 but it is OK in 1.22.1, even with the same configure options.

[Tan Shao Yi]

I notice the OK version of the binary includes libdl and libpthread:

# ldd nginx

linux-vdso.so.1 (0x00007ffc9653c000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f82e70e7000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f82e70e2000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f82e70a6000)
libpam.so.0 => /lib/x86_64-linux-gnu/libpam.so.0 (0x00007f82e7094000)
libmaxminddb.so.0 => /lib/x86_64-linux-gnu/libmaxminddb.so.0 (0x00007f82e708d000)
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007f82e6ff1000)
libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f82e6f5e000)
libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f82e6c00000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f82e6f3f000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f82e6a1f000)
libaudit.so.1 => /lib/x86_64-linux-gnu/libaudit.so.1 (0x00007f82e6f0e000)
/lib64/ld-linux-x86-64.so.2 (0x00007f82e723b000)
libcap-ng.so.0 => /lib/x86_64-linux-gnu/libcap-ng.so.0 (0x00007f82e6f04000)

For some reason, configure and make no longer links it to libpthread.

I recompiled nginx without --threads and that seems to stop the general protection faults.

[Maxim Dounin]

Are you able to reproduce this without 3rd party modules, notably --add-module=../ngx_http_auth_pam_module-master as seen in the nginx -V output you've provided?

The backtrace suggests that there is satisfy any; in your configuration, and the WWW-Authenticate header was added by a 3rd party module which does not take into account header API changes in nginx 1.23.0. See 711737177b77 for details about the particular issue you are seeing, and d26db4f82d7d for the change in general and examples on how to modify the auth_pam module to fix this.

[maxim]

Ticket retargeted after milestone closed

[Tan Shao Yi]

Thank you for your help! Yes, it looks like ngx_http_auth_pam_module-master was the cause of the problem and it's OK now if I do not compile it in.

[Maxim Dounin]

Thanks for confirming, closing this.