From: Glynn Clements (glynn.clementsvirgin.net)
Date: Mon Apr 05 2004 - 17:41:18 CDT

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Rick Zhong wrote:

> I am trying the Alpha One's "Smashing The Stack For Fun And Profit". But i
> realise that the new GCC compiler has add a random padding between the last
> local variable and return address during the memory allocation.
>
> For example:
> void function(int a, int b, int c)
> {
> int z[1];
> }
>
> For for this arrray z, the gcc behave normally for z[1], z[2] , it will
> allocate 4 byte for z[1], and 8 byte for z[2]. When I increase to z[3] and
> z[4] the compiler will allocate 24 bytes. When z[5] is 5, it will allocate
> 40 bytes.
>
> My question is how the compile do the padding? Any specific algorithm?

AFAICT, the size of the stack frame (including the 8 bytes for the
saved EIP and EBP) is always a multiple of 16, except when the local
variables occupy exactly 1, 2, 4 or 8 bytes.

Experimentation suggests that, in the general case, the amount
explicitly subtracted from ESP (not including the pushed EIP or EBP)
is:

        ((n+15)/16)*16+8

where n is the number of bytes which the local variables would
require. IOW, if n isn't a multiple of 16, round it upwards to the
next multiple of 16, then add 8.

The exception is that for n=1, n=2 and n=4, it subtracts 4 bytes from
ESP, while for n=8 it subtracts 8 bytes.

You can confirm this by generating a file comprising functions of the
form:

        void f1(void) { char d[1]; }
        void f2(void) { char d[2]; }
        void f3(void) { char d[3]; }
        ...

compiling it to assembler with "gcc -S", then extract the "subl"
instructions with grep.

--
Glynn Clements <glynn.clementsvirgin.net>

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]