Opened 12 years ago

Closed 9 years ago

#118 closed defect (fixed)

Lack of "Vary" handling in proxy can lead to corrupted downloads

Reported by: Oz Solomon Owned by: somebody
Priority: minor Milestone:
Component: nginx-core Version: 1.0.x
Keywords: Vary, proxy Cc:
uname -a: Linux 2.6.18-274.17.1.el5 #1 SMP Tue Jan 10 17:25:58 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.0.12
built by gcc 4.1.2 20080704 (Red Hat 4.1.2-51)
TLS SNI support disabled
configure arguments: --user=nginx --group=nginx --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/var/run/nginx.pid --lock-path=/var/lock/subsys/nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module --with-http_image_filter_module --with-http_geoip_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_stub_status_module --with-http_perl_module --with-mail --with-file-aio --with-mail_ssl_module --with-ipv6 --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --with-ld-opt=-Wl,-E

Description

Documentation clearly states that the proxy module doesn't handle the Vary response header. However, "not handling" can mean a few things, and currently it means things are broken.

Example:
nginx as reverse proxy to Apache, and Apache is using mod_deflate to compress the content.

User 1 makes a request for content using a modern browser, which nginx passes on to Apache, and Apache happily returns using a compressed response (Content-Encoding: gzip, Vary: Accept-Encoding).

User 2 makes a request for the same content, but using a brain dead browser (in this case IE, which for 0.5% of users does not accept gzip encoding as we've recently measured). Despite the fact that the browser isn't sending "Accept-Encoding: gzip" to the server, the content is served from nginx's cache in compressed format and the browser receives a response it can't handle.

Expected:
nginx should never cache responses that contain the Vary header.

Better Yet, Expected:
nginx should properly handle the Vary header and only serve cached versions if the headers match.

Actual:
Whatever is cached after the first request is served.

Note: For a long form background on this issue, please see: http://www.notthewizard.com/2012/02/27/nginx-reverse-proxy-can-cause-ie-to-fail/

Change History (7)

comment:1 by Maxim Dounin, 12 years ago

Status: newaccepted

Yep, Vary handling needs to be implemented. Just disabling cache for responses with the Vary header might be an acceptable solution.

For now, there are two basic workarounds for the particular issue:

  1. Use proxy_set_header Accept-Encoding ""; to make sure backend won't return compressed content (or just switch off compression on backend), and do a compression on nginx. (Or vice versa, use proxy_set_header Accept-Encoding "gzip"; and use gunzip module to decompress if needed.)
  1. Add $http_accept_encoding to the cache key.

comment:2 by Maxim Dounin, 12 years ago

And, just in case, another option which should work too:

proxy_no_cache $upstream_http_vary;

See proxy_no_cache.

comment:3 by Nils Toedtmann, 11 years ago

Hi, i think i ran into with nginx + fastcgi_cache + php-fpm.

Is this the same bug (if it is one)? Observe that the client does not send any "Accept-Encoding:" header:

GET /membership HTTP/1.1
User-Agent: curl/7.21.6 (i686-pc-linux-gnu) libcurl/7.21.6
OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
Host: islington.the-hub.net
Accept: */*

HTTP/1.1 200 OK
Server: nginx/1.4.1
Date: Thu, 16 May 2013 21:48:25 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.3.10-1ubuntu3.6
Last-Modified: Thu, 01 Jan 1970 00:00:00 GMT
Content-Encoding: gzip
Vary: Accept-Encoding

Is this bug being worked on?

in reply to:  3 comment:4 by Maxim Dounin, 11 years ago

sensitive: 0

Replying to Nils Toedtmann <nils.toedtmann@gmail.com>:

Is this the same bug (if it is one)? Observe that the client does not send any "Accept-Encoding:" header:

This suggests there is a problem on php side, unlrelated to this bug.

comment:5 by Sebastian Krebs, 11 years ago

Sorry, that I push this ticket, but is there a chance to get proper "Vary"-handling with for proxy-/fastcgi-caching?

Version 0, edited 11 years ago by Sebastian Krebs (next)

comment:6 by Maxim Dounin <mdounin@…>, 9 years ago

In 60fde1bc7236691e9218fdfa4c050c1c617aa334/nginx:

Cache: disable caching of responses with Vary (ticket #118).

The "proxy_ignore_header" directive now undersands the "Vary" parameter
to ignore the header as needed.

comment:7 by Maxim Dounin, 9 years ago

Resolution: fixed
Status: acceptedclosed

Fixed by 60fde1bc7236 and following commits.

Note: See TracTickets for help on using tickets.