Opened 8 months ago

Last modified 8 months ago

#2619 new defect

Issues with HTTP/3 Configuration and listen 443 quic reuseport; Directive Affecting Server Block Functionality

Reported by: desaisoftwaree@… Owned by:
Priority: blocker Milestone:
Component: documentation Version: 1.25.x
Keywords: Cc:
uname -a: Linux automore 6.5.0-1015-gcp #15~22.04.1-Ubuntu SMP Wed Feb 14 21:22:00 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.25.4
built by gcc 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)
built with OpenSSL 3.0.2 15 Mar 2022
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_v3_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -ffile-prefix-map=/data/builder/debuild/nginx-1.25.4/debian/debuild-base/nginx-1.25.4=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

Description

Title: Routing and Configuration Issues with HTTP/3 and listen 443 quic reuseport; in Nginx 1.25.4

Environment:

Nginx Version: 1.25.4
Operating System: Ubuntu 22.04.4 LTS
OpenSSL Version: OpenSSL 3.0.2
Description:
I'm experiencing challenges with setting up HTTP/3 on multiple server blocks within the same Nginx instance. The primary issue revolves around the listen 443 quic reuseport; directive, which seems to be the only way to enable HTTP/3 support. Utilizing this directive across multiple server blocks leads to misrouting and other server blocks not functioning correctly under HTTP/3.

Problems:

Limited HTTP/3 Activation Method: The current setup requires listen 443 quic reuseport; for HTTP/3 activation. This limitation restricts how HTTP/3 can be enabled across various server blocks.

Single Block Restriction for QUIC: Incorporating listen 443 quic reuseport; in more than one server block results in configuration issues, limiting HTTP/3 support to a single server block.

Incorrect Subdomain Routing with QUIC: Activating listen 443 quic reuseport; in any server block leads to a significant bug. Nginx does not serve the requested subdomain; instead, it serves content from an incorrect server block, which suggests a routing or server block selection problem with QUIC enabled.

Steps to Reproduce:

Configure multiple server blocks on Nginx 1.25.4, with each serving different subdomains. Ensure each has SSL enabled and aims to support HTTP/3.
In the configurations, enable HTTP/3 by using the directives http3 on;, http2 on;, and listen 443 quic reuseport; for each server block.
Access the subdomains using a client that supports HTTP/3.
Expected Behavior:
Each request to a subdomain should correctly serve content from its corresponding server block configuration, with HTTP/3 enabled for clients that support it.

Actual Behavior:

Only one server block can successfully use listen 443 quic reuseport; without encountering configuration errors.
When listen 443 quic reuseport; is enabled in any server block, Nginx does not serve the content from the requested server block for specific subdomains. Instead, it serves content from a different block, indicating an issue with routing or server block selection when QUIC is enabled.

Configuration:
server {

listen 80;
server_name _;
return 301 ​https://$host$request_uri;

}

server {

listen 443 ssl;
server_name gallery.myapp.app;

http3 on;
http2 on;

quic_retry on;
ssl_early_data on;
add_header Alt-Svc 'h3=":$server_port"; ma=86400';

proxy_intercept_errors on;

ssl_certificate /etc/letsencrypt/live/myapp.app/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/myapp.app/privkey.pem;

location / {

root /home/usr/Ecosystem-App/gallery-server/public/;
index index.html;
try_files $uri $uri.html /index.html =404;

}

error_page 500 502 503 504 /50x.html;
location = /50x.html {

root html;

}

}

server {

listen 443 ssl;
server_name billing.myapp.app;

http3 on;
http2 on;

quic_retry on;
ssl_early_data on;
add_header Alt-Svc 'h3=":$server_port"; ma=86400';

proxy_intercept_errors on;

ssl_certificate /etc/letsencrypt/live/myapp.app/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/myapp.app/privkey.pem;

location / {

root /home/usr/Ecosystem-App/billing-server/public/;
index index.html;
try_files $uri $uri.html /index.html =404;

}

error_page 500 502 503 504 /50x.html;
location = /50x.html {

root html;

}

}

server {

listen 443 ssl;
server_name dev.myapp.app;

http3 on;
http2 on;

quic_retry on;
ssl_early_data on;
add_header Alt-Svc 'h3=":$server_port"; ma=86400';

proxy_intercept_errors on;

ssl_certificate /etc/letsencrypt/live/myapp.app/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/myapp.app/privkey.pem;

location / {

root /home/usr/Ecosystem-App/main-server/public/;
index index.html;
try_files $uri $uri.html /index.html =404;

}

error_page 500 502 503 504 /50x.html;
location = /50x.html {

root html;

}

}

server {

listen 443 ssl;
listen 443 quic reuseport;
server_name myapp.app www.myapp.app app.myapp.app;

http3 on;
http2 on;

quic_retry on;
ssl_early_data on;
add_header Alt-Svc 'h3=":$server_port"; ma=86400';

proxy_intercept_errors on;

ssl_certificate /etc/letsencrypt/live/myapp.app/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/myapp.app/privkey.pem;

location / {

root /home/usr/Ecosystem-App/main-server/public/;
index index.html;
try_files $uri $uri.html /index.html =404;

}

error_page 500 502 503 504 /50x.html;
location = /50x.html {

root html;

}

}

server {

listen 443 ssl;
server_name *.myapp.app;

http3 on;
http2 on;

quic_retry on;
ssl_early_data on;
add_header Alt-Svc 'h3=":$server_port"; ma=86400';

proxy_intercept_errors on;

ssl_certificate /etc/letsencrypt/live/myapp.app/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/myapp.app/privkey.pem;

location / {

root /home/usr/Ecosystem-App/d2c-server/public/;
index index.html;
try_files $uri $uri.html /index.html =404;

}

error_page 500 502 503 504 /50x.html;
location = /50x.html {

root html;

}

}

Additional Information:

No workarounds have been found to enable HTTP/3 across multiple server blocks without encountering the described issues.
This issue significantly impacts the ability to utilize HTTP/3 for enhanced performance across multiple services hosted on the same Nginx instance.
Note: This report has been formatted considering WikiFormatting support for better readability.

Change History (2)

comment:1 by Roman Arutyunyan, 8 months ago

Only one server block can successfully use listen 443 quic reuseport; without encountering configuration errors.

That's the expected behavior. You are only allowed to specify reuseport socket option for one listen. Please use listen 443 quic for others.

When listen 443 quic reuseport; is enabled in any server block, Nginx does not serve the content from the requested server block for specific subdomains. Instead, it serves content from a different block, indicating an issue with routing or server block selection when QUIC is enabled.

nginx should serve content from the server block where the listen directive for the port is specified. If it's not the case please provide more details.

in reply to:  1 comment:2 by desaisoftwaree@…, 8 months ago

Replying to Roman Arutyunyan:

Only one server block can successfully use listen 443 quic reuseport; without encountering configuration errors.

That's the expected behavior. You are only allowed to specify reuseport socket option for one listen. Please use listen 443 quic for others.

When listen 443 quic reuseport; is enabled in any server block, Nginx does not serve the content from the requested server block for specific subdomains. Instead, it serves content from a different block, indicating an issue with routing or server block selection when QUIC is enabled.

nginx should serve content from the server block where the listen directive for the port is specified. If it's not the case please provide more details.

Dear Roman Arutyunyan,

I wanted to extend my sincere gratitude for your insightful comment on the Nginx ticket regarding the listen 443 quic reuseport; directive. Your clarification on the expected behavior and guidance on using listen 443 quic was incredibly helpful. It not only resolved my confusion but also steered me in the right direction for configuring my server blocks more effectively. Your expertise and willingness to share knowledge are greatly appreciated. Thank you for your valuable contribution to the community.

Best regards

Version 0, edited 8 months ago by desaisoftwaree@… (next)
Note: See TracTickets for help on using tickets.