Opened 18 months ago

Closed 18 months ago

Last modified 18 months ago

#1558 closed defect (wontfix)

nginx does not pass metadata to grpc server

Reported by: codeandcookies@… Owned by:
Priority: minor Milestone:
Component: other Version: 1.13.x
Keywords: Cc:
uname -a: Darwin C02NW9YVG3QD 17.5.0 Darwin Kernel Version 17.5.0: Fri Apr 13 19:32:32 PDT 2018; root:xnu-4570.51.2~1/RELEASE_X86_64 x86_64
nginx -V: nginx version: nginx/1.13.12 built by clang 9.1.0 (clang-902.0.39.1) built with OpenSSL 1.0.2o 27 Mar 2018 TLS SNI support enabled configure arguments: --prefix=/usr/local/Cellar/nginx/1.13.12 --sbin-path=/usr/local/Cellar/nginx/1.13.12/bin/nginx --with-cc-opt='-I/usr/local/opt/pcre/include -I/usr/local/opt/openssl/include' --with-ld-opt='-L/usr/local/opt/pcre/lib -L/usr/local/opt/openssl/lib' --conf-path=/usr/local/etc/nginx/nginx.conf --pid-path=/usr/local/var/run/nginx.pid --lock-path=/usr/local/var/run/nginx.lock --http-client-body-temp-path=/usr/local/var/run/nginx/client_body_temp --http-proxy-temp-path=/usr/local/var/run/nginx/proxy_temp --http-fastcgi-temp-path=/usr/local/var/run/nginx/fastcgi_temp --http-uwsgi-temp-path=/usr/local/var/run/nginx/uwsgi_temp --http-scgi-temp-path=/usr/local/var/run/nginx/scgi_temp --http-log-path=/usr/local/var/log/nginx/access.log --error-log-path=/usr/local/var/log/nginx/error.log --with-debug --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-ipv6 --with-mail --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module

Description

nginx is configured with the most basic grpc proxy config. This is the only configuration in use:

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  logs/access.log  main;

    server {
        listen 9080 http2;
        location / {
            grpc_pass localhost:5000;
        }
    }
}

I have a grpc client that adds metadata to its outgoing context:

func testDataInterceptor(ctx context.Context, method string, req, reply interface{}, cc *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
	traceID := uuid.New().String()
	ctx = metadata.AppendToOutgoingContext(ctx,
		"trace.id", traceID,
		"trace.application", "test-client",
		"auth.jwt", xxxxx)

	return invoker(ctx, method, req, reply, cc, opts...)
}

On the server side, this metadata is logged by an interceptor.

	md, ok := metadata.FromIncomingContext(ctx)
	if !ok {
		log.Printf("Attempt to execute %s without providing metadata", info.FullMethod)
		return nil, status.Error(codes.Unauthenticated, "Expected metadata but it was not present in the context")
	}
	log.Println("Context metadata", md)

If the client is communicating directly with the server, the server has access to the metadata, producing following log:

2018/05/22 16:56:36 map[:authority:[localhost:5000] content-type:[application/grpc] user-agent:[grpc-go/1.12.0] trace.id:[7f0d8e18-654d-4b75-bae5-0e3f4ea33058] trace.application:[test-client] auth.jwt:[xxxxxxxx]]

If the client is communicating with nginx, then this information is no longer present when the request reaches the grpc server:

2018/05/22 16:57:10 Context metadata map[:authority:[localhost:5000] content-type:[application/grpc] user-agent:[grpc-go/1.12.0]]

Same behaviour on:

vagrant@scaler-1:~$ uname -a
Linux scaler-1 4.4.0-31-generic #50~14.04.1-Ubuntu SMP Wed Jul 13 01:07:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
vagrant@scaler-1:~$ nginx -V
nginx version: nginx/1.14.0
built by gcc 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.4)
built with OpenSSL 1.0.1f 6 Jan 2014
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

Change History (2)

comment:1 Changed 18 months ago by mdounin

  • Resolution set to wontfix
  • Status changed from new to closed

The problem is that headers you add in your code have dots (.) in them. As such, these headers are ignored by nginx by default. You may either change header names to something without dots, or instruct nginx to accept arbitrary header names by using the ignore_invalid_headers directive:

ignore_invalid_headers off;

Note though that allowing arbitrary header names might result security issues.

comment:2 Changed 18 months ago by codeandcookies@…

That was fast, thanks!
Changed the metadata keys and it works like a charm.

Note: See TracTickets for help on using tickets.