failed (12: Cannot allocate memory) while sending mp4 to client

[Vlad K]

File:

-rw------- 1 web web 2954987161 Jan  9 15:02 /xxxxxx/attachments/31/504/df213e53-926f-4660-94ee-e52a9f6fdca0.mp4

Nginx.conf:

http {
    include            mime.types;
    default_type       application/octet-stream;

    sendfile           off;  # Tried both with sendfile on and off
    server_names_hash_max_size          2048;
    server_names_hash_bucket_size       128;
    gzip                                on;
    gzip_min_length                     1000;
    gzip_buffers                        2000 64k;
    gzip_types                          text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
    gzip_static                         on;
    client_max_body_size                5500m;
    client_body_buffer_size             256k;
    client_header_buffer_size           1k;
    large_client_header_buffers         4 4k;
    output_buffers                      1 32k;
    postpone_output                     1460;
    client_header_timeout               3m;
    client_body_timeout                 3m;
    send_timeout                        3m;
    proxy_connect_timeout               90;
    proxy_send_timeout                  90;
    proxy_read_timeout                  90;
    proxy_buffer_size                   4k;
    proxy_buffers                       4 32k;
    proxy_busy_buffers_size             64k;
    proxy_temp_file_write_size          64k;
    client_body_temp_path  tmp/client_temp;
    proxy_temp_path        tmp/proxy_temp;
    fastcgi_temp_path      tmp/fastcgi_temp;

    server {
        listen       80;
        server_name  localhost;
        access_log   nginx-access.log;
        server_name_in_redirect  off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      ===== cut =====

        location ~ /file/(.+\.mp4)$ {
            sendfile off;
            alias  $alias$1;
            mp4;
            mp4_buffer_size     5m;
            mp4_max_buffer_size 40m;
        }

Request:

- - [11/Jan/2012:20:07:59 +0800] "GET /file/attachments/31/504/df213e53-926f-4660-94ee-e52a9f6fdca0.mp4?start=1000 HTTP/1.0" 200 48979 "-" "Wget/1.12 (linux-gnu)"

Nginx error message:

2012/01/11 20:07:59 [emerg] 15463#0: *1 malloc(18446744072188658253) failed (12: Cannot allocate memory) while sending mp4 to client, client: xxxxx, server: localhost, request: "GET /file/attachments/31/504/df213e53-926f-4660-94ee-e52a9f6fdca0.mp4?start=1000 HTTP/1.0", host: "xxxxx"

OS:

CentOS Linux release 6.0 (Final)

Free:

# free
             total       used       free     shared    buffers     cached
Mem:       2055616    1648424     407192          0     175648    1226188
-/+ buffers/cache:     246588    1809028
Swap:      4128760          0    4128760

[Vlad K]

meantime, request without start option
wget http://servername.com/file/attachments/31/504/df213e53-926f-4660-94ee-e52a9f6fdca0.mp4
works fine.

[Maxim Dounin]

Could you please provide debug log? See ​here for details.

[Maxim Dounin]

Thanks, the problem seems to be clear enough. It looks like the file has invalid metadata and nginx isn't able to handle this correctly. Could you please make the file available somehow for further testing?

[Maxim Dounin]

Hm, I've tried to reproduce the problem by manually corrupting a file here, and I tend to think there is no corruption in the file. Instead, it's just problem with handling of files > 2G (but less than 4G, i.e. using 32bit offsets in mp4).

Could you please check if the following patch fixes things?

--- a/src/http/modules/ngx_http_mp4_module.c
+++ b/src/http/modules/ngx_http_mp4_module.c
@@ -165,10 +165,10 @@ typedef struct {
     ((u_char *) (p))[7] = n4
 
 #define ngx_mp4_get_32value(p)                                                \
-    ( (((u_char *) (p))[0] << 24)                                             \
-    + (((u_char *) (p))[1] << 16)                                             \
-    + (((u_char *) (p))[2] << 8)                                              \
-    + (((u_char *) (p))[3]) )
+    ( ((uint32_t) ((u_char *) (p))[0] << 24)                                  \
+    + (           ((u_char *) (p))[1] << 16)                                  \
+    + (           ((u_char *) (p))[2] << 8)                                   \
+    + (           ((u_char *) (p))[3]) )
 
 #define ngx_mp4_set_32value(p, n)                                             \
     ((u_char *) (p))[0] = (u_char) ((n) >> 24);                               \

[Maxim Dounin]

In [4403/nginx]:

(The changeset message doesn't reference this ticket)

[Maxim Dounin]

Fixed in 1.1.13.

[Vlad K]

yes, it works with nginx-1.1.13

[Maxim Dounin]

In [4442/nginx]:

(The changeset message doesn't reference this ticket)

[Sergey Gorelkin]

Hello.
I've got the same problem with large file on my website ​http://minaevlive.ru
The problem you can see at following video
​http://minaevlive.ru/archive/link981fb761b8/
If you do seek offset in this video, nginx generate a error:
2012/05/31 14:31:00 [emerg] 38336#0: *58 malloc(18446744070009754518) failed (12: Cannot allocate memory) while sending mp4 to client, client: 62.141.97.62,
server: russia2.goodoo.ru, request: "GET /minaevlive/86/hd720p.mp4?start=16819.384251968502&from=russia HTTP/1.1", host: "83.222.2.205"

File size about 4G
Link to origin file: ​http://83.222.2.205/minaevlive/86/hd720p.mp4
Config:

location ~ \.mp4$ {

expires -1;
mp4_buffer_size 1m;
mp4_max_buffer_size 40m;
access_log /logs/nginx/access.log main;
mp4;

}

[root@russia2 ~]# nginx -V
nginx version: nginx/1.2.0
built by gcc 4.2.1 20070719 [FreeBSD]
TLS SNI support disabled
configure arguments: --prefix=/usr/local/nginx --with-cc-opt='-I /usr/local/include' --with-ld-opt='-L /usr/local/lib' --conf-path=/usr/local/nginx/conf/nginx.conf --sbin-path=/usr/local/sbin/nginx --pid-path=/var/run/nginx.pid --error-log-path=/var/log/nginx-error.log --user=www --group=www --with-ipv6 --http-client-body-temp-path=/var/tmp/nginx/client_body_temp --http-proxy-temp-path=/var/tmp/nginx/proxy_temp --http-log-path=/var/log/nginx-access.log --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_ssl_module --with-http_stub_status_module --with-pcre
[root@russia2 ~]# uname -a
FreeBSD russia2.goodoo.ru 7.2-RELEASE FreeBSD 7.2-RELEASE #0: Fri May 1 07:18:07 UTC 2009 root@…:/usr/obj/usr/src/sys/GENERIC amd64

[Maxim Dounin]

The file in question (​http://83.222.2.205/minaevlive/86/hd720p.mp4) is corrupted, it claims mdat atom length to be 0x04c1200f (79765519 bytes):

00a92330  05 68 74 cd 05 68 76 e2  04 c1 20 0f 6d 64 61 74  |.ht..hv... .mdat|
00a92340  00 00 00 01 04 c1 20 0f  00 03 78 4b 65 88 84 37  |...... ...xKe..7|

Obviously this isn't correct and disagree with real file size (over 4G) as well as real mdat atom size and and other information in the moov atom, which in turn causes problems with streaming.