Memory overrun due to alignment issues when cross-compiling mingw on linux

[Orgad Shaneh]

Hi,

When I cross-compile nginx for Windows on linux host using mxe, NGX_ALIGNMENT is set to 16. This is set in auto/os/conf:

case "$NGX_MACHINE" in
...
    *)
        have=NGX_ALIGNMENT value=16 . auto/define
        NGX_MACH_CACHE_LINE=32
    ;;
esac

But the pool is allocated using plain malloc (on Windows target), which is *not* aligned to 16 bytes (it's aligned to 4), so pool->d.end is 4-byte aligned.

Now, in ngx_palloc_small, the following code causes integer underflow, and the comparison doesn't work as expected:

        m = p->d.last;

        if (align) {
            m = ngx_align_ptr(m, NGX_ALIGNMENT);
        }

        if ((size_t) (p->d.end - m) >= size) { /* <------- This condition */
            p->d.last = m + size;

            return m;
        }

Since m is aligned, and end is not, end - m can be a negative number. Then it is cast to size_t, which is unsigned, and so it just keeps writing past the memory pool, and overruns the next pool.

A minimal patch that solves this issue is:

src/core/ngx_palloc.c

@@ -160 +160 @@
             m = ngx_align_ptr(m, NGX_ALIGNMENT);
         }
 
-        if ((size_t) (p->d.end - m) >= size) {
+        if ((intptr_t) (p->d.end - m) >= (intptr_t)size) {
             p->d.last = m + size;
 
             return m;

But it would be better to assign the right alignment for Windows, even when cross-building. And also ensure that end is aligned correctly.

I also noticed that the default alignment when not set is sizeof(long), which is wrong for Win64. It should be sizeof(intptr_t).

Build command:

CC=i686-w64-mingw32.static-gcc ./auto/configure --crossbuild=win32 --without-http_rewrite_module --without-http_gzip_module

$ grep -1 ALIGN objs/ngx_auto_config.h

#ifndef NGX_ALIGNMENT
#define NGX_ALIGNMENT  16
#endif