Re: [PATCH] oom03: use size_t for memory length to fix 32-bit overflow

Hi all,

The alloc_mem() function is supposed to test out of memory conditions. How it works is it calls mmap() on a giant chunk of memory. It's either LENGTH (2GB) or "TESTMEM * 2 + TST_MB" (3GB) bytes. This mmap() is generally supposed to succeed. Then at the bottom of the alloc_mem() function when we actually try to use all the memory, the thread is supposed to die with a SIGKILL.

The problem is that length is signed so on a 32-bit system it will be negative. That means that at the bottom of the function when we loop through the memory, the for loop is a no-op and there is no SIGKILL. Fix this by changing the type to size_t which is unsigned.

I dared to change subject to oom.h and merged. Thanks!

Kind regards, Petr