Opened 12 years ago

Closed 3 years ago

Last modified 20 months ago

#196 closed defect (fixed)

Inconsistent behavior on uri's with unencoded spaces followed by H

Reported by: Matt Kolve Owned by: somebody
Priority: minor Milestone:
Component: nginx-core Version: 1.3.x
Keywords: Cc:
uname -a: Darwin mango 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386 i386 MacBookPro8,1 Darwin
nginx -V: nginx version: nginx/1.3.4
built by gcc 4.2.1 (Apple Inc. build 5666) (dot 3)
TLS SNI support enabled
configure arguments: --prefix=/opt/local --with-cc-opt='-I/opt/local/include -O2' --with-ld-opt=-L/opt/local/lib --conf-path=/opt/local/etc/nginx/nginx.conf --error-log-path=/opt/local/var/log/nginx/error.log --http-log-path=/opt/local/var/log/nginx/access.log --pid-path=/opt/local/var/run/nginx/nginx.pid --lock-path=/opt/local/var/run/nginx/nginx.lock --http-client-body-temp-path=/opt/local/var/run/nginx/client_body_temp --http-proxy-temp-path=/opt/local/var/run/nginx/proxy_temp --http-fastcgi-temp-path=/opt/local/var/run/nginx/fastcgi_temp --http-uwsgi-temp-path=/opt/local/var/run/nginx/uwsgi_temp --with-ipv6 --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-google_perftools_module --with-http_gzip_static_module --with-http_mp4_module --with-http_ssl_module --with-http_stub_status_module --with-debug

Description

When requesting files with unencoded spaces, nginx will typically respond with the file requested. But if the filename has a space followed by a capital H, nginx responds with a 400 error.

[foo@bar Downloads]$ nc -vv 127.0.0.1 8000
Ncat: Version 6.01 ( http://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:8000.
GET /t h HTTP/1.1
Host: 127.0.0.1:8000

HTTP/1.1 200 OK
Server: nginx/1.3.4
Date: Sun, 12 Aug 2012 20:22:30 GMT
Content-Type: application/octet-stream
Content-Length: 4
Last-Modified: Sun, 12 Aug 2012 18:30:35 GMT
Connection: keep-alive
ETag: "5027f64b-4"
Accept-Ranges: bytes

bar

[foo@bar Downloads]$ nc -vv 127.0.0.1 8000
Ncat: Version 6.01 ( http://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:8000.
GET /a H HTTP/1.1
<html>
<head><title>400 Bad Request</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx/1.3.4</center>
</body>
</html>
Ncat: 18 bytes sent, 172 bytes received in 7.29 seconds.
[foo@bar Downloads]$ nc -vv 127.0.0.1 8000
Ncat: Version 6.01 ( http://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:8000.
GET /a%20H HTTP/1.1
Host: 127.0.0.1:8000

HTTP/1.1 200 OK
Server: nginx/1.3.4
Date: Sun, 12 Aug 2012 20:23:32 GMT
Content-Type: application/octet-stream
Content-Length: 4
Last-Modified: Sun, 12 Aug 2012 18:34:44 GMT
Connection: keep-alive
ETag: "5027f744-4"
Accept-Ranges: bytes

bar

Change History (15)

comment:1 by Maxim Dounin, 12 years ago

Do you have a problem with some particular client?

Just for history, original change to allow space in uri was introduced to please broken clients, see http://mailman.nginx.org/pipermail/nginx/2010-June/020878.html. Note well that the current behaviour isn't compatible well Apache one (Apache will assume anything after space is a protocol, not a part of the uri, and "GET /t h HTTP/1.0" will return "/t" file).

comment:2 by Matt Kolve, 12 years ago

The issue is with XBMC using nginx-dav-ext-module. It looks like nginx-dav-ext-module doesn't send back properly encoded uri's in the PROPFIND xml.

<D:href>/foo/a test</D:href>

XBMC then takes these unencoded uri's and tries to use them without encoding, which is where the error with nginx seems to have come from.

Apache's webdav sends back the href encoded properly

<D:href>/foo/a%20test</D:href>

Does nginx use 'H' as the delimiter between a uri and protocol?
Should http clients encode uri's if the uri's haven't been encoded in html or xml? I think Chrome and Firefox do this, but am not sure if the rfc specifies to do this.

Last edited 12 years ago by Matt Kolve (previous) (diff)

in reply to:  2 comment:3 by Maxim Dounin, 12 years ago

Status: newaccepted

The issue is with XBMC using nginx-dav-ext-module. It looks like nginx-dav-ext-module doesn't send back properly encoded uri's in the PROPFIND xml.

This looks like a problem in dav-ext module (it should escape href), and probably minor problem in XBMC (as it failed to create http request properly incorrectly assuming presented uri is valid). You may want to report these problems to XBMC and dav-ext authors.

Does nginx use 'H' as the delimiter between a uri and protocol?

As per RFC it uses space as a delimiter, but backtracks to uri parsing (to please broken clients) if it doesn't see 'H' after space.

Should http clients encode uri's if the uri's haven't been encoded in html or xml? I think Chrome and Firefox do this, but am not sure if the rfc specifies to do this.

HTTP clients should not sent syntactically invalid http requests, so two valid options are: 1) escape or 2) reject link as broken.

I tend to think that proper fix in nginx would be to disallow spaces in uri by default, this should make such problems much easier to find.

comment:4 by Mohammad Alsaleh, 11 years ago

I tend to think that proper fix in nginx would be to disallow spaces in uri by default, this should make such problems much easier to find.

Agreed.

Whatever is decided. NGINX should return a proper 400 error response with invalid requests. Not a headerless reply with an error message embedded in html.

comment:5 by Maxim Dounin, 11 years ago

sensitive: 0

The response is a proper HTTP/0.9 response, which is used as nginx wasn't able to find proper HTTP/1.0 (or later) request, and defaults to HTTP/0.9.

comment:6 by https://stackoverflow.com/users/573152/bernard-rosset, 8 years ago

Still there in nginx 1.8.0

comment:7 by Maxim Dounin, 7 years ago

See also #1371.

comment:8 by Valentin V. Bartenev, 5 years ago

See also #1728.

comment:9 by bruno9779@…, 4 years ago

user@nginx01:~$ nginx -V
nginx version: nginx/1.10.3 (Ubuntu)
built with OpenSSL 1.0.2g  1 Mar 2016
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_secure_link_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module --with-stream --with-stream_ssl_module --with-threads --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/headers-more-nginx-module --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/nginx-auth-pam --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/nginx-cache-purge --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/nginx-dav-ext-module --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/nginx-development-kit --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/nginx-echo --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/ngx-fancyindex --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/nginx-http-push --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/nginx-lua --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/nginx-upload-progress --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/nginx-upstream-fair --add-module=/build/nginx-U6uhdy/nginx-1.10.3/debian/modules/ngx_http_substitutions_filter_module
user@nginx01:~$ uname -a
Linux nginx01.int.us-east-1.escm.co 4.4.0-159-generic #187-Ubuntu SMP Thu Aug 1 16:28:06 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Still present in nginx 1.10.3

Last edited 4 years ago by bruno9779@… (previous) (diff)

comment:10 by Maxim Dounin, 4 years ago

uname -a: modified (diff)

For the record, this also seems to cause RTSP requests being interpreted as HTTP/0.9, leading to HTTP/0.9 responses (see this thread and my response).

comment:11 by Maxim Dounin, 4 years ago

uname -a: modified (diff)

See also #1985.

comment:12 by Maxim Dounin <mdounin@…>, 3 years ago

In 7881:52338ddf9e2f/nginx:

Disabled spaces in URIs (ticket #196).

From now on, requests with spaces in URIs are immediately rejected rather
than allowed. Spaces were allowed in 31e9677b15a1 (0.8.41) to handle bad
clients. It is believed that now this behaviour causes more harm than
good.

comment:13 by Maxim Dounin, 3 years ago

Resolution: fixed
Status: acceptedclosed

comment:14 by Maxim Dounin, 3 years ago

See also #2241.

comment:15 by Maxim Dounin, 20 months ago

See also #2373.

Note: See TracTickets for help on using tickets.