broken header while reading PROXY protocol in nginx stream with pass module

[sad3rasd@…]

the provided configuration should accept a proxy protocol header encapsulated in a ssl connection but it doesnt work
when the server block try to read and parse proxy protocol header it fail with the error:
broken header: "A%��м�P���S�V]Җƨ���Tp����@o$HB����~F��r�K3��8^{Q������W�M�" while reading PROXY protocol, client: ..., server: unix:/run/nginx/uds_pp.sock}

i tried to preread the stream with njs and it start with the correct proxy protocol header

i also noticed that if no ssl is involved (server unix:/run/nginx/uds.sock without ssl) everything works as expected

relevant part of nginx.conf

stream {
    ...
    map $ssl_preread_server_name $us {
        "example.com" unix:/run/nginx/example.sock;
        default unix:/run/nginx/uds.sock;
    }
    server {
        listen 8443;
        ssl_preread on;
        pass $us;
    }
    server {
        listen unix:/run/nginx/uds.sock ssl;
        ssl_certificate /etc/nginx/self-signed.pem;
        ssl_certificate_key /etc/nginx/self-signed.key;
        ssl_session_timeout 1d;
        ssl_session_cache shared:MozSSL:10m;
        ssl_session_tickets off;
        ssl_protocols TLSv1.3;
        ssl_prefer_server_ciphers off;
        pass unix:/run/nginx/uds_pp.sock;
    }
    server {
        listen unix:/run/nginx/uds_pp.sock proxy_protocol;
        proxy_pass ...;
        ...
    }
}

[Roman Arutyunyan]

Thanks for reporting this. Indeed PROXY protocol incapsulated in SSL cannot be used with pass due to using recv(MSG_PEEK) on the socket. So far I don't see a simple fix for this. This can be avoided by using proxy_pass instead.

Meanwhile, PROXY protocol is not supposed to be incapsulated in another protocol in the first place.

[sad3rasd@…]

Thank you for your reply.
I did two further tests after my ticket:

• with proxy_pass that work like you said
• move the server block inside an http block (and proxy_pass stuff inside a location block) and it does work with pass directive (see configuration below).

nginx.conf

stream {
    ...
    map $ssl_preread_server_name $us {
        "example.com" unix:/run/nginx/example.sock;
        default unix:/run/nginx/uds.sock;
    }
    server {
        listen 8443;
        ssl_preread on;
        pass $us;
    }
    server {
        listen unix:/run/nginx/uds.sock ssl;
        ssl_certificate /etc/nginx/self-signed.pem;
        ssl_certificate_key /etc/nginx/self-signed.key;
        ssl_session_timeout 1d;
        ssl_session_cache shared:MozSSL:10m;
        ssl_session_tickets off;
        ssl_protocols TLSv1.3;
        ssl_prefer_server_ciphers off;
        pass unix:/run/nginx/uds_pp.sock;
    }
}
http {
    ...
    server {
        listen unix:/run/nginx/uds_pp.sock proxy_protocol;
        location / {
            proxy_pass ...;
            ...
        }
    }
}

why this different behavior between stream to stream and stream to http? can be handled in the same way in order to fix the issue?

Regards

[Roman Arutyunyan]

Indeed there's a slightly different behavior in http and stream regarding reading client PROXY protocol header. It originates from the fact that in plaintext HTTP nginx reads client request in a buffer and then parses PROXY protocol header at the start of the buffer. In stream client protocol is unknown and nginx avoids reading data beyond PROXY protocol header. And since there's no way to tell in advance how big the header will be, it first MSG_PEEKs the input, and only then the actual header is read out from the socket. The MSG_PEEK part is incompatible with an SSL socket being used.

[sad3rasd@…]

Got it, I'll use proxy_pass for now, hoping for a fix in the future

Thank you again Roman