additional headers not sent when directory index is forbidden

[https://stackoverflow.com/users/1164131/niko-s-p]

running nginx in docker (nginx:mainline, currently 1.19.0) to serve static files, I have added the usual set of headers via add_headers like this:

server {
  listen                *:80 default_server;
  server_name           _;
  server_tokens         off;

  add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" always;
  add_header Content-Security-Policy "default-src 'self'; style-src 'self' 'unsafe-hashes' 'unsafe-inline';";
  add_header Referrer-Policy strict-origin;
  add_header X-Content-Type-Options nosniff;
  add_header X-Frame-Options SAMEORIGIN;
  add_header X-XSS-Protection 1;

  location / {
      root   /usr/share/nginx/html;
      index  index.html index.htm;
  }
}

When a request is made to a path without index file, directory listing is denied (rightfully so) and a 403 status is returned.
When this happens, none of the extra headers are returned.
These additional headers should always be returned, it makes us fail security certifications because automated scanners find pages without the proper headers set.

While I don't have an example at hand, I could imagine that there is a scenario where being able to circumvent additional headers during a request in this way might enable or at least aid some kind of malicious action.

[Maxim Dounin]

According to your configuration, the Strict-Transport-Security header is expected to be returned along with 403 responses. Testing with the provided configuration suggests this is what indeed happens:

$ curl -I http://127.0.0.1:8080/
HTTP/1.1 403 Forbidden
Server: nginx
Date: Sun, 16 Aug 2020 12:55:54 GMT
Content-Type: text/html
Content-Length: 146
Connection: keep-alive
Strict-Transport-Security: max-age=15768000; includeSubDomains; preload

If you observe different behaviour, please demonstrate your testing results and provide full configuration as shown by nginx -T and a ​debugging log.

[https://stackoverflow.com/users/1164131/niko-s-p]

You are correct, the HSTS header is indeed returned, I am sorry, I misstated earlier.
To be accurate, out of the additional headers set in the configuration example above, only the HSTS header is returned, none of the other headers are.

Full configuration

root@851c2d2a97e7:/# nginx -T
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

# configuration file /etc/nginx/mime.types:

types {
    text/html                                        html htm shtml;
    text/css                                         css;
    text/xml                                         xml;
    image/gif                                        gif;
    image/jpeg                                       jpeg jpg;
    application/javascript                           js;
    application/atom+xml                             atom;
    application/rss+xml                              rss;

    text/mathml                                      mml;
    text/plain                                       txt;
    text/vnd.sun.j2me.app-descriptor                 jad;
    text/vnd.wap.wml                                 wml;
    text/x-component                                 htc;

    image/png                                        png;
    image/svg+xml                                    svg svgz;
    image/tiff                                       tif tiff;
    image/vnd.wap.wbmp                               wbmp;
    image/webp                                       webp;
    image/x-icon                                     ico;
    image/x-jng                                      jng;
    image/x-ms-bmp                                   bmp;

    font/woff                                        woff;
    font/woff2                                       woff2;

    application/java-archive                         jar war ear;
    application/json                                 json;
    application/mac-binhex40                         hqx;
    application/msword                               doc;
    application/pdf                                  pdf;
    application/postscript                           ps eps ai;
    application/rtf                                  rtf;
    application/vnd.apple.mpegurl                    m3u8;
    application/vnd.google-earth.kml+xml             kml;
    application/vnd.google-earth.kmz                 kmz;
    application/vnd.ms-excel                         xls;
    application/vnd.ms-fontobject                    eot;
    application/vnd.ms-powerpoint                    ppt;
    application/vnd.oasis.opendocument.graphics      odg;
    application/vnd.oasis.opendocument.presentation  odp;
    application/vnd.oasis.opendocument.spreadsheet   ods;
    application/vnd.oasis.opendocument.text          odt;
    application/vnd.openxmlformats-officedocument.presentationml.presentation
                                                     pptx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
                                                     xlsx;
    application/vnd.openxmlformats-officedocument.wordprocessingml.document
                                                     docx;
    application/vnd.wap.wmlc                         wmlc;
    application/x-7z-compressed                      7z;
    application/x-cocoa                              cco;
    application/x-java-archive-diff                  jardiff;
    application/x-java-jnlp-file                     jnlp;
    application/x-makeself                           run;
    application/x-perl                               pl pm;
    application/x-pilot                              prc pdb;
    application/x-rar-compressed                     rar;
    application/x-redhat-package-manager             rpm;
    application/x-sea                                sea;
    application/x-shockwave-flash                    swf;
    application/x-stuffit                            sit;
    application/x-tcl                                tcl tk;
    application/x-x509-ca-cert                       der pem crt;
    application/x-xpinstall                          xpi;
    application/xhtml+xml                            xhtml;
    application/xspf+xml                             xspf;
    application/zip                                  zip;

    application/octet-stream                         bin exe dll;
    application/octet-stream                         deb;
    application/octet-stream                         dmg;
    application/octet-stream                         iso img;
    application/octet-stream                         msi msp msm;

    audio/midi                                       mid midi kar;
    audio/mpeg                                       mp3;
    audio/ogg                                        ogg;
    audio/x-m4a                                      m4a;
    audio/x-realaudio                                ra;

    video/3gpp                                       3gpp 3gp;
    video/mp2t                                       ts;
    video/mp4                                        mp4;
    video/mpeg                                       mpeg mpg;
    video/quicktime                                  mov;
    video/webm                                       webm;
    video/x-flv                                      flv;
    video/x-m4v                                      m4v;
    video/x-mng                                      mng;
    video/x-ms-asf                                   asx asf;
    video/x-ms-wmv                                   wmv;
    video/x-msvideo                                  avi;
}

# configuration file /etc/nginx/conf.d/headers.conf:
add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload" always;
add_header Content-Security-Policy "default-src 'self'; style-src 'self' 'unsafe-hashes' 'unsafe-inline';";
add_header Referrer-Policy strict-origin;
add_header X-Content-Type-Options nosniff;
add_header X-Frame-Options SAMEORIGIN;
add_header X-XSS-Protection 1;

# configuration file /etc/nginx/conf.d/static.conf:
server {
  listen                *:80 default_server;
  server_name           _;
  server_tokens         off;
  return 301 https://$host$request_uri;
}
server {
  listen                *:443 ssl http2;
  server_name           _;
  keepalive_timeout     70;
  server_tokens         off;

  ssl_certificate       /etc/ssl/private/cert.pem;
  ssl_certificate_key   /etc/ssl/private/cert.key;
  ssl_protocols TLSv1.2;
  ssl_prefer_server_ciphers on;
  ssl_ciphers "EECDH+AESGCM EDH+AESGCM";

 include conf.d/headers.conf;

  location / {
      root   /usr/share/nginx/html;
      index  index.html index.htm;
  }
}

root@851c2d2a97e7:/# nginx -v
nginx version: nginx/1.17.1
root@851c2d2a97e7:/# nginx -V
nginx version: nginx/1.17.1
built by gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
built with OpenSSL 1.1.0j  20 Nov 2018
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.17.1/debian/debuild-base/nginx-1.17.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

This is how I am testing it:
the docker compose file:

version: '3'

services:
  static:
    image: nginx:mainline
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./nginx:/etc/nginx/conf.d:ro
      - ./data/:/usr/share/nginx/html:ro
      - ./certs:/etc/ssl/private
    restart: unless-stopped
    logging:
      driver: json-file
      options:
        max-size: 60m
        max-file: '3'

# ls -lah data
total 8
drwxr-xr-x  3 xxx  staff    96B Aug  9 10:38 .
drwxr-xr-x  8 xxx  staff   256B Aug 17 07:58 ..

# curl -Ik https://127.0.0.1:443
HTTP/2 403
server: nginx
date: Mon, 17 Aug 2020 04:00:09 GMT
content-type: text/html
content-length: 146
strict-transport-security: max-age=15768000; includeSubDomains; preload

# curl -Ik https://127.0.0.1:443/index.html
HTTP/2 404
server: nginx
date: Mon, 17 Aug 2020 04:00:19 GMT
content-type: text/html
content-length: 146
strict-transport-security: max-age=15768000; includeSubDomains; preload

# touch data/index.html
# curl -Ik https://127.0.0.1:443/index.html
HTTP/2 200
server: nginx
date: Mon, 17 Aug 2020 04:00:30 GMT
content-type: text/html
content-length: 0
last-modified: Mon, 17 Aug 2020 04:00:27 GMT
etag: "5f3a00db-0"
strict-transport-security: max-age=15768000; includeSubDomains; preload
content-security-policy: default-src 'self'; img-src 'self' data:; style-src 'self' 'unsafe-hashes' 'unsafe-inline'; font-src 'self' https://fonts.gstatic.com/; script-src 'self' 'unsafe-eval' 'nonce-cGxsY29va2ll' 'nonce-bGluZXM9Ng==' 'nonce-bGluZXM9MQ==' 'nonce-bGluZXM9Mg==' 'nonce-bGluZXM9Mw==' 'nonce-bGluZXM9NA==' 'nonce-bGluZXM9NQ==' 'sha256-ej1ubIZijTeUIPla4O/JQOd/aWayaKhMc8lZgpn+Z9I=' 'sha256-Y8mVD+nkH7csuKPBhYdvkYi0p8Mka4VQW6+xhvu90iA=';
referrer-policy: strict-origin
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1
accept-ranges: bytes

If you still need the debug log I can get the nginx:mainline Dockerfile and change the entrypoint to run nginx-debug, I think using the information above it should be reproducible fine though using docker.

[Maxim Dounin]

You are correct, the HSTS header is indeed returned, I am sorry, I misstated earlier.
To be accurate, out of the additional headers set in the configuration example above, only the HSTS header is returned, none of the other headers are.

Quoting the ​add_header directive description:

Adds the specified field to a response header provided that the response code equals 200, 201 (1.3.10), 204, 206, 301, 302, 303, 304, 307 (1.1.16, 1.0.13), or 308 (1.13.0).
...
If the always parameter is specified (1.7.5), the header field will be added regardless of the response code.

Only the HSTS header in your configuration is marked by the always parameter and hence returned along with 403 responses.

[https://stackoverflow.com/users/1164131/niko-s-p]

I am sorry to have wasted your time.
Usually it is me who tells others to RTFM..