crash at __kernel_vsyscall() doesn't occur after GCC optimization disabled

I had a nasty crash occuring in my application. GDB always backtraced it to __kernel_vsyscall(). I couldn't find anything suspicious in source code after debugging.

But randomly disabling once '-O3' optimisation flag in GCC compiler seemed to resolve the issue. I am not sure if that is the reason of crash or if compiler might have done someth开发者_JAVA百科ing nasty during optimisation. Any comment or information will be helpful.

In some backtraces shown below, the only erroneous thing observed in application code is the length of the buffer received from MsgQ (buffLen). But the source code makes sure that maximum size of messages sent and received over MsgQ is 2048 bytes. Couldn't trace why and when the length returned by msgrcv() call could have got corrupted.

CRASH 1:

1. 0x00110416 in __kernel_vsyscall ()
2. 0x00352391 in send () from /lib/libc.so.6
3. 0x0034d31c in __vsyslog_chk () from /lib/libc.so.6
4. 0x0034d5b7 in syslog () from /lib/libc.so.6
5. 0x0804f08f in procType1Msg(MsgBuff=0xbfc10275)
6. 0x080498c5 in procRcvdMsQBuf(buffLen=134515184,buff='value optimized out')
7. main (argc=3, argv=0xbfc10b24)

CRASH 2:

1. 0x00110416 in __kernel_vsyscall ()
2. 0x00352391 in send () from /lib/libc.so.6
3. 0x0034d31c in __vsyslog_chk () from /lib/libc.so.6
4. 0x0034d5b7 in syslog () from /lib/libc.so.6
5. 0x0806234b in dumpMsg (buff=0xbfe01832 "U\006\" x,¨n,#®U\027\bI@\024",len=23)
6. 0x0804e539 in procType1Msg(MsgBuff=0xbfe01815)
7. 0x0804956d in procRcvdMsQBuf(buffLen=134515040,buff='value optimized out')
8. main (argc=3, argv=0xbfe020c4)

CRASH 3:

1. 0x00110416 in __kernel_vsyscall ()
2. 0x00353e3f in msgrcv () from /lib/libc.so.6
3. 0x080639b9 in getMsgQBuffer(msg_id=196611, pMsg=0xbfa9d360, lMsgType=0, piErrorNo=0xbfa9d35c)
4. 0x080497dd in main (argc=Cannot access memory at address 0x30003)

My guess is that you have a memory corruption (probably buffer overflow) somewhere in your code.

When you compile with level 3 optimization the code the compiler outputs is such that the buffer overruns writes over something important (maybe corrupts the stack?) and it so happens that the non optimized code produced by the compiler when running with no optimization flag is different so the overflow runs over something else and does not cause this specific symptom. The bug is probably still there and it may manifest itself in other ways or even not at all - until you change something non related and then it will bite you again.

The __kernel_vsyscall() thing is simply the a glibc function that is called internally whenever you do a syscall. There is nothing significant there.

My recommendation: run your program under valgrind. It will most likely find the memory overrun for you.