Opened 4 years ago

Closed 4 years ago

#948 closed defect (fixed)

Web DAV module strange behaviour when client resumes upload

Reported by: yf13@… Owned by:
Priority: minor Milestone:
Component: nginx-module Version: 1.4.x
Keywords: webDAV Cc:
uname -a: Linux ubuntu 3.13.0-55-generic #94-Ubuntu SMP Thu Jun 18 00:27:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
nginx -V: nginx version: nginx/1.4.6 (Ubuntu)
built by gcc 4.8.2 (Ubuntu 4.8.2-19ubuntu1)
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_secure_link_module --with-http_spdy_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/headers-more-nginx-module --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/nginx-auth-pam --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/nginx-cache-purge --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/nginx-dav-ext-module --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/nginx-development-kit --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/nginx-echo --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/ngx-fancyindex --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/nginx-http-push --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/nginx-lua --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/nginx-upload-progress --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/nginx-upstream-fair --add-module=/build/nginx-BU0ZJt/nginx-1.4.6/debian/modules/ngx_http_substitutions_filter_module

Description

With nginx web dav module, we can't resume a broken file upload correctly. Instead, the partial file on server was overwritten and only the last part was left on server finally.

To reproduce this:

1 prepare fileA, fileB, then let fileC = fileA + fileB (cat fileA fileB>fileC);
2 use curl -T fileA localhost/dav/fileA to upload the first part (which is same as fileA);
3 use curl -C lenA -T fileA fileC localhost/dav/fileA to upload the 2nd part only from the broken point
4 check server side "fileA", it is same as fileB now, but fileC is expected.

This blocks us from using web dav module as a way for resumable file uploading.


Change History (6)

comment:1 by Maxim Dounin, 4 years ago

Resuming PUT requests using the Content-Range header as curl does is not something nginx supports. The fact that file is silently broken looks like a bug though. RFC 2616 explicitly required to return 501 if Content-Range is present but not understand by the server:

   The recipient of the entity MUST NOT ignore any Content-*
   (e.g. Content-Range) headers that it does not understand or implement
   and MUST return a 501 (Not Implemented) response in such cases.

Newer RFC 7231 seems to assume that Content-Range in requests is always an error and suggests to return 400 instead:

   An origin server that allows PUT on a given target resource MUST send
   a 400 (Bad Request) response to a PUT request that contains a
   Content-Range header field (Section 4.2 of [RFC7233]), since the
   payload is likely to be partial content that has been mistakenly PUT
   as a full representation.  Partial content updates are possible by
   targeting a separately identified resource with state that overlaps a
   portion of the larger resource, or by using a different method that
   has been specifically defined for partial updates (for example, the
   PATCH method defined in [RFC5789]).

Some additional discussion can be found here (comments to the second answer are particularly interesting).

Here is a patch that ensures that nginx will return an error (501, as RFC 2616 approach looks more correct for me than RFC 7231 clearly incorrect "is likely" vision):

# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1460127002 -10800
#      Fri Apr 08 17:50:02 2016 +0300
# Node ID b4c747531d6b795558cf20e7cfa04bbda58fc043
# Parent  a1193af52b37caf6c7c7c518f6dd24d24934a552
Dav: return 501 on PUT with ranges (ticket #948).

diff --git a/src/http/modules/ngx_http_dav_module.c b/src/http/modules/ngx_http_dav_module.c
--- a/src/http/modules/ngx_http_dav_module.c
+++ b/src/http/modules/ngx_http_dav_module.c
@@ -161,6 +161,12 @@ ngx_http_dav_handler(ngx_http_request_t 
             return NGX_HTTP_CONFLICT;
         }
 
+        if (r->headers_in.content_range) {
+            ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                          "PUT with range is unsupported");
+            return NGX_HTTP_NOT_IMPLEMENTED;
+        }
+
         r->request_body_in_file_only = 1;
         r->request_body_in_persistent_file = 1;
         r->request_body_in_clean_file = 1;
diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c
+++ b/src/http/ngx_http_request.c
@@ -110,6 +110,10 @@ ngx_http_header_t  ngx_http_headers_in[]
                  offsetof(ngx_http_headers_in_t, content_length),
                  ngx_http_process_unique_header_line },
 
+    { ngx_string("Content-Range"),
+                 offsetof(ngx_http_headers_in_t, content_range),
+                 ngx_http_process_unique_header_line },
+
     { ngx_string("Content-Type"),
                  offsetof(ngx_http_headers_in_t, content_type),
                  ngx_http_process_header_line },
diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -182,6 +182,7 @@ typedef struct {
     ngx_table_elt_t                  *user_agent;
     ngx_table_elt_t                  *referer;
     ngx_table_elt_t                  *content_length;
+    ngx_table_elt_t                  *content_range;
     ngx_table_elt_t                  *content_type;
 
     ngx_table_elt_t                  *range;

comment:2 by yf13@…, 4 years ago

Maxim,

Thanks for the quick reply... the discussions are really interesting as you mentioned.

I understand this patch can protect the partial file from being overwritten, this is nice. :)

But is there any way to achieve resumable upload after the patch with curl tool? Should one use HTTP PATCH?

Regards,
Yanfeng

comment:3 by Maxim Dounin, 4 years ago

As I said in the very first sentence of comment:1, resuming PUT requests is not something nginx supports. And PATCH isn't supported either, and as far as I understand, it is also not supported by curl. There are no plans to add support, though feel free to suggest a patch if you are willing to implement it.

comment:4 by Maxim Dounin, 4 years ago

Priority: blockerminor
Status: newaccepted

comment:5 by Maxim Dounin <mdounin@…>, 4 years ago

In 6538:055cbb52ac1d/nginx:

Dav: return 501 on PUT with ranges (ticket #948).

comment:6 by Maxim Dounin, 4 years ago

Resolution: fixed
Status: acceptedclosed

Fix committed, thanks for reporting this.

Note: See TracTickets for help on using tickets.