How does the operating system detect a stack overflow?

On many operating systems the stack and heap begin at opposite sides of a process's virtual address space and grow toward one another. This allows the stack to expand as much as possible without hitting the heap.

Suppose that I have a program that cause开发者_C百科s a stack overflow. My current understanding is that this would result in the stack growing uncontrollably toward the heap and eventually hitting it. Is this correct? If it is, how does the operating system detect that a stack overflow occurs? It seems like the OS wouldn't be able to detect that the program was trying to use virtual memory allocated for the heap as part of the stack, since they'd be in contiguous memory regions.

I know that this is operating-system specific, but insight on the mechanism by which this occurs in any operating system would definitely be helpful. This has been bugging me for a while and I can't seem to find any good explanations.

The OS allocates some space to the stack. When the process accesses an unallocated part of the stack, a page fault is raised by the processor and caught by the OS. If the OS believes it's still reasonable to grow the stack, it simply allocates new space for it and returns control to the process. If it's not reasonable, a stack overflow exception is raised.

This is just intuition, but making sure that the stack doesn't interfere with the heap sounds like the job of the JVM. I see no reason why I couldn't make my own terrible programming language where I let the stack begin to overwrite the heap (before crashing).

Most modern processors have MMU (memory manegement unit) and it is "hardware" device to apply virtual address to each program and each program lives in its own memory space. OS handle this MMU and when program want to read or write on address out of dedicated memory MMU raises interrupt.

http://en.wikipedia.org/wiki/Memory_management_unit

This way it is pretty straitgfoward how it might detect SO. Stack and Heap are not on continuous memory addresss.

I have also seen different approach using guard words. Stack for each process was allocated on heap and filled whit guard wards (something like 0xc0cac01a). This way it was easy to get size of stack for each process just by counting guard words. Operating system raised panic if there were not at least one guard word.

Stack overflows go backwards in the stack -- they work by overwriting data in already initialized parts of the stack, which is possible precisely because stacks grow downwards.

Thus, the operating system cannot detect this condition, as the process is allowed to modify the initialized parts of its stack at any time it wants.

Extending the stack works by the process using the stack space, and the OS trapping into the page fault handler because no page has been set up. Some OSes allow these accesses only on a "guard page", i.e. the page immediately before the current stack will trigger reallocation, others look at the contents of the stack pointer register at the time of the fault to see whether this was supposed to be a stack access.