Web DAV module strange behaviour when client resumes upload

[yf13@…]

With nginx web dav module, we can't resume a broken file upload correctly. Instead, the partial file on server was overwritten and only the last part was left on server finally.

To reproduce this:

1 prepare fileA, fileB, then let fileC = fileA + fileB (cat fileA fileB>fileC);
2 use curl -T fileA localhost/dav/fileA to upload the first part (which is same as fileA);
3 use curl -C lenA -T fileA fileC localhost/dav/fileA to upload the 2nd part only from the broken point
4 check server side "fileA", it is same as fileB now, but fileC is expected.

This blocks us from using web dav module as a way for resumable file uploading.

[Maxim Dounin]

Resuming PUT requests using the Content-Range header as curl does is not something nginx supports. The fact that file is silently broken looks like a bug though. ​RFC 2616 explicitly required to return 501 if Content-Range is present but not understand by the server:

   The recipient of the entity MUST NOT ignore any Content-*
   (e.g. Content-Range) headers that it does not understand or implement
   and MUST return a 501 (Not Implemented) response in such cases.

Newer ​RFC 7231 seems to assume that Content-Range in requests is always an error and suggests to return 400 instead:

   An origin server that allows PUT on a given target resource MUST send
   a 400 (Bad Request) response to a PUT request that contains a
   Content-Range header field (Section 4.2 of [RFC7233]), since the
   payload is likely to be partial content that has been mistakenly PUT
   as a full representation.  Partial content updates are possible by
   targeting a separately identified resource with state that overlaps a
   portion of the larger resource, or by using a different method that
   has been specifically defined for partial updates (for example, the
   PATCH method defined in [RFC5789]).

Some additional discussion can be found ​here (comments to the second answer are particularly interesting).

Here is a patch that ensures that nginx will return an error (501, as RFC 2616 approach looks more correct for me than RFC 7231 clearly incorrect "is likely" vision):

# HG changeset patch
# User Maxim Dounin <mdounin@mdounin.ru>
# Date 1460127002 -10800
#      Fri Apr 08 17:50:02 2016 +0300
# Node ID b4c747531d6b795558cf20e7cfa04bbda58fc043
# Parent  a1193af52b37caf6c7c7c518f6dd24d24934a552
Dav: return 501 on PUT with ranges (ticket #948).

diff --git a/src/http/modules/ngx_http_dav_module.c b/src/http/modules/ngx_http_dav_module.c
--- a/src/http/modules/ngx_http_dav_module.c
+++ b/src/http/modules/ngx_http_dav_module.c
@@ -161,6 +161,12 @@ ngx_http_dav_handler(ngx_http_request_t 
             return NGX_HTTP_CONFLICT;
         }
 
+        if (r->headers_in.content_range) {
+            ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                          "PUT with range is unsupported");
+            return NGX_HTTP_NOT_IMPLEMENTED;
+        }
+
         r->request_body_in_file_only = 1;
         r->request_body_in_persistent_file = 1;
         r->request_body_in_clean_file = 1;
diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c
+++ b/src/http/ngx_http_request.c
@@ -110,6 +110,10 @@ ngx_http_header_t  ngx_http_headers_in[]
                  offsetof(ngx_http_headers_in_t, content_length),
                  ngx_http_process_unique_header_line },
 
+    { ngx_string("Content-Range"),
+                 offsetof(ngx_http_headers_in_t, content_range),
+                 ngx_http_process_unique_header_line },
+
     { ngx_string("Content-Type"),
                  offsetof(ngx_http_headers_in_t, content_type),
                  ngx_http_process_header_line },
diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -182,6 +182,7 @@ typedef struct {
     ngx_table_elt_t                  *user_agent;
     ngx_table_elt_t                  *referer;
     ngx_table_elt_t                  *content_length;
+    ngx_table_elt_t                  *content_range;
     ngx_table_elt_t                  *content_type;
 
     ngx_table_elt_t                  *range;

[yf13@…]

Maxim,

Thanks for the quick reply... the discussions are really interesting as you mentioned.

I understand this patch can protect the partial file from being overwritten, this is nice. :)

But is there any way to achieve resumable upload after the patch with curl tool? Should one use HTTP PATCH?

Regards,
Yanfeng

[Maxim Dounin]

As I said in the very first sentence of comment:1, resuming PUT requests is not something nginx supports. And PATCH isn't supported either, and as far as I understand, it is also not supported by curl. There are no plans to add support, though feel free to ​suggest a patch if you are willing to implement it.

[Maxim Dounin <mdounin@…>]

In 6538:055cbb52ac1d/nginx:

Dav: return 501 on PUT with ranges (ticket #948).

[Maxim Dounin]

Fix committed, thanks for reporting this.