Opened 8 months ago

Last modified 8 months ago

#2625 new defect

nginx proxy_pass variable DNS resolution not updated when there is another proxy_pass with same domain and without variable — at Version 1

Reported by: lkgendev@… Owned by:
Priority: minor Milestone:
Component: nginx-module Version: 1.14.x
Keywords: proxy_pass dns Cc: lkgendev@…
uname -a: Linux dba-tlv-wkrlxy 4.18.0-477.10.1.el8_8.x86_64 #1 SMP Wed Apr 5 13:35:01 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.14.1
built by gcc 8.2.1 20180905 (Red Hat 8.2.1-3) (GCC)
built with OpenSSL 1.1.1 FIPS 11 Sep 2018 (running with OpenSSL 1.1.1k FIPS 25 Mar 2021)
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-http_auth_request_module --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-debug --with-cc-opt='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'

Description (last modified by lkgendev@…)

according to the documentation of nginx dns disccovery, when using proxy_pass with a variable for the domain, the variable should be reresolved upon DNS change of IP
however when there are two proxy_pass locations, one with variable and another without variable for the same domain name, it appears that the proxy_pass with variable does not re-resolves the DNS IP changes of the domain name and stays with the IP at the beginning even if DNS changed the domain IP

To reproduce on Redhat 8

  1. install nginx: sudo yum install nginx
  2. start local dns that can read /etc/hosts entries: sudo systemctl start systemd-resolved
  3. modify the nginx configuration file /etc/nginx/nginx.conf

3a. add to the log_format $upstream_addr e.g.

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for" "$upstream_addr"';

3b. add two proxy_pass, one with variable resolver of 127.0.0.53 with valid of 1 second and the other without variable

        location /test1 {
            resolver 127.0.0.53 valid=1s;
            set $testbackend https://test.local;
            proxy_pass $testbackend;
        }

        location /test2 {
            proxy_pass https://test.local;
        }
  1. add to /etc/hosts an entry for test.local pointing to 127.0.0.200

127.0.0.200 test.local

  1. use dig to check the DNS resolution of test.local: dig @127.0.0.53 test.local

check IP is resolved to 127.0.0.200

  1. restart nginx for configuration to take effect: sudo systemctl start nginx
  2. use curl to connect to the test1 location: curl localhost:80/test1
  3. check nginx access log last lines, there should be upstream address of 127.0.0.200:443

tail -20 /var/log/nginx/access.log
...
::1 - - [04/Apr/2024:15:07:46 +0300] "GET /test1 HTTP/1.1" 502 4020 "-" "curl/7.61.1" "-" "127.0.0.200:443"
upstram IP is 127.0.0.200

  1. change the /etc/hosts entry for test.local to have a different IP 127.0.0.201 instead of the previous 127.0.0.200

127.0.0.201 test.local

  1. use dig to check the DNS resolution of test.local: dig @127.0.0.53 test.local

check IP is resolved to 127.0.0.201

  1. wait for 30 seconds so the valid time for the DNS entry should expire for the resolver
  2. use curl to connect to the test1 location: curl localhost:80/test1
  3. check nginx access log last lines, the upstream address should change to 127.0.0.201:443 but it will remain as before

tail -20 /var/log/nginx/access.log
...
::1 - - [04/Apr/2024:15:17:27 +0300] "GET /test1 HTTP/1.1" 502 4020 "-" "curl/7.61.1" "-" "127.0.0.200:443"

Change History (2)

by lkgendev@…, 8 months ago

Attachment: nginx.conf added

nginx configuration to reproduce the DNS resolution problem with domain name test.local

comment:1 by lkgendev@…, 8 months ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.