Opened 5 years ago

Closed 5 years ago

Last modified 4 years ago

#821 closed defect (fixed)

Header "Expires" and "Cache-Control" is not sent for random requests when using "expires" directive

Reported by: wkuranowski@… Owned by:
Priority: major Milestone:
Component: nginx-core Version: 1.8.x
Keywords: expires header missing Cc:
uname -a: Linux front-1.img.srv 2.6.32-573.7.1.el6.x86_64 #1 SMP Tue Sep 22 22:00:00 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.8.0
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
built with OpenSSL 1.0.1e-fips 11 Feb 2013
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_auth_request_module --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 --with-http_spdy_module --with-cc-opt='-O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic'

Description (last modified by Maxim Dounin)

There is a problem with missing "Expires" and "Cache-Control" headers from my upstream "nfs_files". It's a rare situation and it looks like "expires" directive is not executed for some reason. I am not able to reproduce it manually, but when I grep nginx cache directory on a production server I can easily find files without those headers. Accessing those files with a browser also confirms that headers are missing. But header "Access-Control-Allow-Origin" is always present.

It happens for all kind of filetypes - css/js/jpg/png, gzipped/not gzipped etc.

I have configuration like below. I hope that I included all relevant config options. I am using nginx 1.8.0 but it happens also on 1.9.4. Error logs are empty.

http {
   sendfile on;

   proxy_cache_lock    on;
   proxy_cache_path    /mnt/nginx-cache levels=1:2 keys_zone=IMG:256m inactive=3h max_size=8g use_temp_path=off;
   proxy_cache_use_stale    error timeout invalid_header updating http_500 http_502 http_503 http_504;

   map $uri $expires {
      default 1d;
      ~^/admin/config/ 10s;
      ~*\.(?:js|css)$ 5m;
   }

   upstream nfs_files {
      server unix:/var/run/nginx_nfs_files.socket;
   }

   server {
      listen 80 default_server;

      location / {
         proxy_cache   IMG;
         #expires header from nfs_files upstream has a higher priority
         proxy_cache_valid   200 1d;
         proxy_pass    http://nfs_files;
      }
   }

   server {
      listen  unix:/var/run/nginx_nfs_files.socket;

      gzip       on;
      gzip_types application/javascript text/css text/xml text/plain application/json;
      gzip_vary  on;

      location / {
         add_header Access-Control-Allow-Origin *;

         #doesn't work for random requests
         expires $expires;

         root    /mnt/nfs/data;
      }
   }
}

Attachments (3)

missing_expires.txt (12.0 KB ) - added by wkuranowski@… 5 years ago.
Missing Expires header
correct_expires.txt (11.9 KB ) - added by wkuranowski@… 5 years ago.
Correct Expires header
missing_gzip_missing_expires.txt (9.2 KB ) - added by wkuranowski@… 5 years ago.
Missing Gzip and Expires

Download all attachments as: .zip

Change History (14)

comment:1 by Maxim Dounin, 5 years ago

Description: modified (diff)

Please provide debug log of a request without Expires header added to the response.

comment:2 by wkuranowski@…, 5 years ago

I have a new hint - I am unable to find missing Expires header when using "expires 12h" instead of "expires $expires". It looks like using variable triggers this issue.

I need a few hours of our production traffic to find request with missing Expires. For now I will try to enable debug log only for a unix socket communication with:

events {
    debug_connection unix:;
}

I hope this will give us some useful data.

by wkuranowski@…, 5 years ago

Attachment: missing_expires.txt added

Missing Expires header

by wkuranowski@…, 5 years ago

Attachment: correct_expires.txt added

Correct Expires header

comment:3 by wkuranowski@…, 5 years ago

I have captured responses without Expires header. Let me know if you need more information.

by wkuranowski@…, 5 years ago

Missing Gzip and Expires

comment:4 by wkuranowski@…, 5 years ago

I have found another interesting example. There is no Expires header and also gzip filter was not executed for a text/css file with "Accept-Encoding: gzip" header set.

File attached.

comment:5 by wkuranowski@…, 5 years ago

Ok, I think that missing gzip filter is because of "Via" header...

comment:6 by wkuranowski@…, 5 years ago

I also see that a request in file "missing_expires.txt" has "cache-control: max-age=0". But this is a request header and should be irrelevant. In "missing_gzip_missing_expires.txt" file there is no "cache-control" and Expires is missing too.

comment:7 by Maxim Dounin, 5 years ago

Please try the following patch (problem found by Roman Arutyunyan):

diff --git a/src/core/ngx_parse.c b/src/core/ngx_parse.c
--- a/src/core/ngx_parse.c
+++ b/src/core/ngx_parse.c
@@ -188,7 +188,7 @@ ngx_parse_time(ngx_str_t *line, ngx_uint
             break;
 
         case 'm':
-            if (*p == 's') {
+            if (p < last && *p == 's') {
                 if (is_sec || step >= st_msec) {
                     return NGX_ERROR;
                 }

in reply to:  7 comment:8 by wkuranowski@…, 5 years ago

I can confirm that your patch solves this issue. I am also unable to reproduce this when I change 5m to 300s in expires map.

Thanks!

comment:9 by Maxim Dounin <mdounin@…>, 5 years ago

In 6287:4ccb37b04454/nginx:

Fixed ngx_parse_time() out of bounds access (ticket #821).

The code failed to ensure that "s" is within the buffer passed for
parsing when checking for "ms", and this resulted in unexpected errors when
parsing non-null-terminated strings with trailing "m". The bug manifested
itself when the expires directive was used with variables.

Found by Roman Arutyunyan.

comment:10 by Maxim Dounin, 5 years ago

Resolution: fixed
Status: newclosed

Fix committed, thanks for reporting the problem and testing the patch.

comment:11 by Maxim Dounin <mdounin@…>, 4 years ago

In 6344:a8ecb0a2193f/nginx:

Fixed ngx_parse_time() out of bounds access (ticket #821).

The code failed to ensure that "s" is within the buffer passed for
parsing when checking for "ms", and this resulted in unexpected errors when
parsing non-null-terminated strings with trailing "m". The bug manifested
itself when the expires directive was used with variables.

Found by Roman Arutyunyan.

Note: See TracTickets for help on using tickets.