Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#650 closed defect (worksforme)

proxy_protocol in virtual hosts

Reported by: Gaurav Purandare Owned by:
Priority: minor Milestone: 1.7
Component: nginx-core Version: 1.6.x
Keywords: proxy_protocol Cc: aamod.nerurkar@…
uname -a:
nginx -V: nginx version: nginx/1.6.1
built by gcc 4.8.2 (Ubuntu 4.8.2-19ubuntu1)
configure arguments: --prefix=/opt/nginx --user=nginx --group=nginx --with-http_spdy_module --without-http_scgi_module --without-http_uwsgi_module --without-http_fastcgi_module --with-pcre --with-http_realip_module --add-module=/opt/stuff/headers-more-nginx-module-0.25 --with-http_dav_module --add-module=./custom_modules/nginx-eval-module-master --add-module=./custom_modules/redis2-nginx-module-0.10

Description

When multiple virtual hosts listen to proxy_protocol as follows:

listen 80 proxy_protocol; #Port 80 may differ per virtual host

Its been observed that, very first virtual host defined in configuration file throws "400 - Bad Request" error for all of the requests received. And, consecutive virtual hosts tend to work normally.

A temporary solution for this issue is, to define a separate default virtual host which listens to proxy_protocol, like below:

server {
listen       80 proxy_protocol;
server_name _; #Default virtual host
}

Its not clear why only very first virtual host throws "400 - Bad Request" error.

Change History (11)

comment:1 by Roman Arutyunyan, 6 years ago

Could you provide more details or even a simple small config for the issue?
With my simple configuration I don't see this problem.
Please watch error.log for errors as well.

comment:2 by Gaurav Purandare, 6 years ago

@arut, did you check the configuration with proxy_protocol enabled on AWS ELB and nginx running behind it?

comment:3 by Roman Arutyunyan, 6 years ago

No, I tested it manually. Does this issue occur only behind AWS ELB?

comment:4 by Gaurav Purandare, 6 years ago

I tested it several times with nginx + AWS ELB - got this issue.

However, you need to enable proxy protocol on ELB explicitly.
Reference: http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html

You can find the complete use-case here:
http://www.raweng.com/blog/2014/11/11/websockets-on-aws-elb

Thanks!

comment:5 by Roman Arutyunyan, 6 years ago

I tried nginx+aws/elb, but it's still fine. Could you provide error log?

error_log logs/error.log debug;

comment:6 by Andrei Belov, 6 years ago

We've just checked with a similar configuration using live ELB with proxy_protocol turned on, and haven't noticed anything wrong.

We've seen a couple of 400 errors shortly after applying proxy_protocol policy to the ELB, while nginx has been using old configuration (listen on 80/tcp without "proxy_protocol" parameter turned on) and ELB was using HTTP health checks, but this was expected.

After adding "proxy_protocol" to the listen directive, reloading nginx configuration and switching ELB health checks to use TCP we were not able to see 400 errors.

Is there a reliable way to reproduce your issue?

How your ELB health checks are configured?

comment:7 by Gaurav Purandare, 6 years ago

@arut, @Andrei - Have you guys tried with configuration that involves default virtual host? The problem I faced was totally specific to very first virtual host. I believe it can be reproduced if you do not define default virtual host, and check the logs for very first virtual host listening to proxy_protocol on nginx instance running behind ELB.

@Andrei, you might want to try this out in non-production environment, instead of production.

When default virtual host is defined, all '400 - Bad requests' errors are seen in that default virtual host. i.e. when default virtual host is defined, I can use rest of the virtual hosts just fine.

I'll also check on my side, and share the error logs once I get chance.

@Andrei, yes - TCP health checks for ELB.

Thanks for your help!

comment:8 by Andrei Belov, 6 years ago

I tried with a number of virtual hosts defined with valid server names - wasn't able to reproduce.

One note though - in the post you've mentioned there is an error in the following configuration snippet:

#Virtual host accepting SSL traffic

  server {
  listen       81 proxy_protocol;
  ...

If you're going to handle SSL on 81/tcp port (I assume you are, looking at the ELB policies), you should add the "ssl" parameter to the "listen" directive, and also specify paths to certificate and key, e.g.:

#Virtual host accepting SSL traffic

  server {
  listen       81 proxy_protocol ssl;
  ssl_certificate your_cert.pem;
  ssl_certificate_key your_cert.key;
  ...

Otherwise, you'll get 400 error and some garbage in access log on every HTTPS request to ELB endpoint.

To summarize - we still do not have a reliable way to reproduce the issue you described.

Please share your relevant error log entries and the whole configuration to make us able to see what happen.

comment:9 by Gaurav Purandare, 6 years ago

@Andrei, sure, will share the error log and configuration.

No, port 81 doesn't use SSL. SSL termination occurs at ELB on port 443 and requests are forwarded to instance port 81.

comment:10 by Andrei Belov, 6 years ago

Resolution: worksforme
Status: newclosed

Unable to reproduce without the entire configuration from the requestor.

comment:11 by Felix Bünemann, 6 years ago

I can reproduce the problem. It happens if there are multiple SSL vHosts using SNI listening on the same IP:Port and not all of them have proxy_protocol enabled.

Last edited 6 years ago by Felix Bünemann (previous) (diff)
Note: See TracTickets for help on using tickets.