id summary reporter owner description type status priority milestone component version resolution keywords cc uname nginx_version 974 Resolver fails if IPv6 DNS records are not returned Kevin Decker "Attempting to utilize Weave DNS to do container resolution. When using the startup-based resolution and `upstream` definitions, the server would launch successfully with all of the standard caveats around existing at startup, etc. {{{ upstream staging { server server-staging.weave.local:3000; } server { location @proxy { proxy_pass http://staging; } } }}} Unfortunately when trying to do the same setup using the resolver, this would not resolve for these instances, but would for global such a google.com. {{{ resolver 172.17.0.1:53; server { location @proxy { set $staging ""server-staging.weave.local:3000""; proxy_pass http://$staging; } } }}} When enabling error logs, the only useful output was: {{{ 2016/05/05 08:42:56 [error] 14#14: *1 server-staging.weave.local could not be resolved (3: Host not found), client: private.ip, server: public.server, request: ""GET / HTTP/1.1"", host: ""public.server"" }}} Eventually, by looking at the full debug logs, I was able to determine that the issue was this record did not have a IPv6 entry. {{{ 2016/05/05 08:42:56 [debug] 14#14: resolve: ""server-staging.weave.local"" 2016/05/05 08:42:56 [debug] 14#14: resolve: ""server-staging.weave.local"" A 53493 2016/05/05 08:42:56 [debug] 14#14: resolve: ""server-staging.weave.local"" AAAA 12359 2016/05/05 08:42:56 [debug] 14#14: resolver DNS response 53493 fl:8580 1/1/0/0 2016/05/05 08:42:56 [debug] 14#14: resolver DNS response qt:1 cl:1 2016/05/05 08:42:56 [debug] 14#14: resolver qs:server-staging.weave.local 2016/05/05 08:42:56 [debug] 14#14: resolver naddrs:1 cname:0000000000000000 ttl:1 2016/05/05 08:42:56 [debug] 14#14: resolver DNS response 12359 fl:8183 1/0/0/0 2016/05/05 08:42:56 [debug] 14#14: resolver DNS response qt:28 cl:1 2016/05/05 08:42:56 [debug] 14#14: resolver qs:server-staging.weave.local 2016/05/05 08:42:56 [debug] 14#14: *1 http upstream resolve: ""/?"" 2016/05/05 08:42:56 [error] 14#14: *1 server-staging.weave.local could not be resolved (3: Host not found), client: private.ip, server: public.ip, request: ""GET / HTTP/1.1"", host: ""public.ip"" }}} vs. google.com: {{{ 2016/05/05 08:39:36 [debug] 10#10: resolve: ""google.com"" 2016/05/05 08:39:36 [debug] 10#10: resolve: ""google.com"" A 42052 2016/05/05 08:39:36 [debug] 10#10: resolve: ""google.com"" AAAA 26761 2016/05/05 08:39:36 [debug] 10#10: resolver DNS response 26761 fl:8180 1/1/0/0 2016/05/05 08:39:36 [debug] 10#10: resolver DNS response qt:28 cl:1 2016/05/05 08:39:36 [debug] 10#10: resolver qs:google.com 2016/05/05 08:39:36 [debug] 10#10: resolver naddrs:1 cname:0000000000000000 ttl:293 2016/05/05 08:39:36 [debug] 10#10: resolver DNS response 42052 fl:8180 1/1/0/0 2016/05/05 08:39:36 [debug] 10#10: resolver DNS response qt:1 cl:1 2016/05/05 08:39:36 [debug] 10#10: resolver qs:google.com 2016/05/05 08:39:36 [debug] 10#10: resolver naddrs:1 cname:0000000000000000 ttl:156 2016/05/05 08:39:36 [debug] 10#10: *1 http upstream resolve: ""/?"" 2016/05/05 08:39:36 [debug] 10#10: *1 name was resolved to 172.217.1.14 2016/05/05 08:39:36 [debug] 10#10: *1 name was resolved to 2607:f8b0:4004:809::200e }}} I have little experience with IPv6 at this point, so I'm not sure what the proper behavior is here. Ideally these records would not be rejected, but if the best practice is for them to be, it would be a huge time saver if the error logging would be expanded to state that the rejection was due to a partial result. Workaround/Fix for end user: Add ipv6=off to Weave resolver definition. {{{ resolver 172.17.0.1:53 ipv6=off; }}} " enhancement closed minor other 1.9.x invalid Linux nginx.weave.local 4.4.5-15.26.amzn1.x86_64 #1 SMP Wed Mar 16 17:15:34 UTC 2016 x86_64 GNU/Linux "nginx version: nginx/1.9.15 built by gcc 4.9.2 (Debian 4.9.2-10) built with OpenSSL 1.0.1k 8 Jan 2015 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_auth_request_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_perl_module=dynamic --add-dynamic-module=debian/extra/njs-1c50334fbea6/nginx --with-threads --with-stream --with-stream_ssl_module --with-http_slice_module --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 --with-http_v2_module --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-z,relro -Wl,--as-needed'"