Opened 15 months ago

Last modified 4 days ago

#118 accepted defect

Lack of "Vary" handling in proxy can lead to corrupted downloads

Reported by: Oz Solomon Owned by: somebody
Priority: minor Milestone:
Component: nginx-core Version: 1.0.x
Keywords: Vary, proxy Cc:
Sensitive: no
uname -a: Linux 2.6.18-274.17.1.el5 #1 SMP Tue Jan 10 17:25:58 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.0.12 built by gcc 4.1.2 20080704 (Red Hat 4.1.2-51) TLS SNI support disabled configure arguments: --user=nginx --group=nginx --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/var/run/nginx.pid --lock-path=/var/lock/subsys/nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module --with-http_image_filter_module --with-http_geoip_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_stub_status_module --with-http_perl_module --with-mail --with-file-aio --with-mail_ssl_module --with-ipv6 --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --with-ld-opt=-Wl,-E

Description

Documentation clearly states that the proxy module doesn't handle the Vary response header. However, "not handling" can mean a few things, and currently it means things are broken.

Example:
nginx as reverse proxy to Apache, and Apache is using mod_deflate to compress the content.

User 1 makes a request for content using a modern browser, which nginx passes on to Apache, and Apache happily returns using a compressed response (Content-Encoding: gzip, Vary: Accept-Encoding).

User 2 makes a request for the same content, but using a brain dead browser (in this case IE, which for 0.5% of users does not accept gzip encoding as we've recently measured). Despite the fact that the browser isn't sending "Accept-Encoding: gzip" to the server, the content is served from nginx's cache in compressed format and the browser receives a response it can't handle.

Expected:
nginx should never cache responses that contain the Vary header.

Better Yet, Expected:
nginx should properly handle the Vary header and only serve cached versions if the headers match.

Actual:
Whatever is cached after the first request is served.

Note: For a long form background on this issue, please see: ​http://www.notthewizard.com/2012/02/27/nginx-reverse-proxy-can-cause-ie-to-fail/

Change History (4)

comment:1 Changed 15 months ago by Maxim Dounin

  • Status changed from new to accepted

Yep, Vary handling needs to be implemented. Just disabling cache for responses with the Vary header might be an acceptable solution.

For now, there are two basic workarounds for the particular issue:

  1. Use proxy_set_header Accept-Encoding ""; to make sure backend won't return compressed content (or just switch off compression on backend), and do a compression on nginx. (Or vice versa, use proxy_set_header Accept-Encoding "gzip"; and use ​gunzip module to decompress if needed.)
  1. Add $http_accept_encoding to the ​cache key.

comment:2 Changed 15 months ago by Maxim Dounin

And, just in case, another option which should work too:

proxy_no_cache $upstream_http_vary;

See ​proxy_no_cache.

comment:3 follow-up: Changed 7 days ago by Nils Toedtmann

Hi, i think i ran into with nginx + fastcgi_cache + php-fpm.

Is this the same bug (if it is one)? Observe that the client does not send any "Accept-Encoding:" header:

GET /membership HTTP/1.1
User-Agent: curl/7.21.6 (i686-pc-linux-gnu) libcurl/7.21.6
OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
Host: islington.the-hub.net
Accept: */*

HTTP/1.1 200 OK
Server: nginx/1.4.1
Date: Thu, 16 May 2013 21:48:25 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.3.10-1ubuntu3.6
Last-Modified: Thu, 01 Jan 1970 00:00:00 GMT
Content-Encoding: gzip
Vary: Accept-Encoding

Is this bug being worked on?

comment:4 in reply to: ↑ 3 Changed 4 days ago by Maxim Dounin

  • Sensitive unset

Replying to Nils Toedtmann <nils.toedtmann@gmail.com>:

Is this the same bug (if it is one)? Observe that the client does not send any "Accept-Encoding:" header:

This suggests there is a problem on php side, unlrelated to this bug.

Note: See TracTickets for help on using tickets.