WebDAV module didn't convert UTF8 encode url into GBK on Windows

[ztyzbb@…]

I'm using Nginx as a static resource server on Windows 10. And I use php_curl to transfer file from Apache to Nginx by HTTP PUT method. When URL include Chinese in UTF8 codec like "/0/%E6%B5%B7%E6%8A%A5%E8%83%8C%E6%99%AF.jpg", the WebDAV just create a file with the UTF-8 string. But Windows use GBK to store Chinese file name. So the UTF-8 string was used as a GBK string without conversion, which lead to an incorrect name on Windows like 海报背景.jpg(correct) => 娴锋姤鑳屾櫙.jpg(incorrect).
If I use GBK encode URL "/0/%BA%A3%B1%A8%B1%B3%BE%B0.jpg"(same as the UTF-8 one in Chinese), every thing works fine, the file name on Windows is correct.
If I try to GET the GBK encode URL, the Nginx return 500 with error log: 1113: No mapping for the Unicode character exists in the target multi-byte code page. And if I use the UTF-8 encode URL, it works fine. So I can figure out the Nginx handle the GET method with a codec conversion, but PUT method not, maybe other WebDAV method not as well.

[ztyzbb@…]

charset has been set to utf-8 in nginx.conf.

[ztyzbb@…]

nginx -V:
nginx version: nginx/1.12.1
built by cl 16.00.40219.01 for 80x86
built with OpenSSL 1.0.2l 25 May 2017
TLS SNI support enabled
configure arguments: --with-cc=cl --builddir=objs.msvc8 --with-debug --prefix= --conf-path=conf/nginx.conf --pid-path=logs/nginx.pid --http-log-path=logs/access.log --error-log-path=logs/error.log --sbin-path=nginx.exe --http-client-body-temp-path=temp/client_body_temp --http-proxy-temp-path=temp/proxy_temp --http-fastcgi-temp-path=temp/fastcgi_temp --http-scgi-temp-path=temp/scgi_temp --http-uwsgi-temp-path=temp/uwsgi_temp --with-cc-opt=-DFD_SETSIZE=1024 --with-pcre=objs.msvc8/lib/pcre-8.40 --with-zlib=objs.msvc8/lib/zlib-1.2.11 --with-select_module --with-http_v2_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_stub_status_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_random_index_module --with-http_secure_link_module --with-http_slice_module --with-mail --with-stream --with-openssl=objs.msvc8/lib/openssl-1.0.2l --with-openssl-opt=no-asm --with-http_ssl_module --with-mail_ssl_module --with-stream_ssl_module

[Maxim Dounin]

See also #458.

[Maxim Dounin]

For the record, I've submitted a ​patch series to address various issues with non-ASCII names on Windows, including this one. In particular, the following patches are directly related to this ticket:

​https://mailman.nginx.org/pipermail/nginx-devel/2023-January/TPQUESKQOUEFDAZXWSH2QW6TFXYDUZI2.html
​https://mailman.nginx.org/pipermail/nginx-devel/2023-January/ZML72O2YVFBSGY2IYFDNWWS2PPKCRNYD.html
​https://mailman.nginx.org/pipermail/nginx-devel/2023-January/ZUR3KNDSRG7W4NRLWH7TGZOBXVGTQIE3.html
​https://mailman.nginx.org/pipermail/nginx-devel/2023-January/GJ47O4REVLDFZT7S36LXMUYOZSSWPCDM.html
​https://mailman.nginx.org/pipermail/nginx-devel/2023-January/7ZQD26HURFGSD3BHIC2ZYJYHE5X537VL.html

Review and testing appreciated.

[Maxim Dounin <mdounin@…>]

In 8133:e0f385521c79/nginx:

Win32: non-ASCII directory names support in ngx_create_dir().

This makes it possible to create directories under prefix with non-ASCII
characters, as well as makes it possible to create directories with non-ASCII
characters when using the dav module (ticket #1433).

To ensure that the dav module operations are restricted similarly to
other file operations (in particular, short names are not allowed), the
ngx_win32_check_filename() function is used. It improved to support
checking of just dirname, and now can be used to check paths when creating
files or directories.

[Maxim Dounin <mdounin@…>]

In 8134:7d60e4584d9e/nginx:

Win32: non-ASCII directory names support in ngx_delete_dir().

This makes it possible to delete directories with non-ASCII characters
when using the dav module (ticket #1433).

[Maxim Dounin <mdounin@…>]

In 8136:37a184966ab3/nginx:

Win32: reworked ngx_win32_rename_file() to use nginx wrappers.

This ensures that ngx_win32_rename_file() will support non-ASCII names
when supported by the wrappers.

Notably, this is used by PUT requests in the dav module when overwriting
existing files with non-ASCII names (ticket #1433).

[Maxim Dounin <mdounin@…>]

In 8137:4b06186670ed/nginx:

Win32: non-ASCII names support in ngx_delete_file().

This makes it possible to delete files with non-ASCII characters
when using the dav module (ticket #1433).

[Maxim Dounin <mdounin@…>]

In 8138:96d894b38667/nginx:

Win32: non-ASCII names support in ngx_rename_file().

This makes it possible to upload files with non-ASCII characters
when using the dav module (ticket #1433).

[Maxim Dounin]

Fix committed and available in nginx 1.23.4 / 1.24.0, closing this. Thanks to all involved.