Illegal read/write error when making legacy code x64 compliant
I have the following MyType::Is_Inst ()
function which is throwing an 开发者_JAVA百科invalid memory access error on return in 64-bit mode but not in 32-bit:
MyType MyType::Is_Inst () {
unsigned char Bar=0;
MyType Foo={0};
return Foo;
}
Looking at the disassembly + step-through, the program crashes at the line
mov dword ptr [rax],ecx
...when the program is basically trying to dereference the original value of %rdx (from when the function was first called), which is now in %rax. However, %rdx is just junk left over from the previous function call.
The last time I had an issue like this, it was because I was missing some compilation flags or the like. Are there any settings I should be aware of for x64 unmanaged c++ projects? Are there any other reasons I might see this behavior?
I can post more of the disassembly if you need it.
The class definition for MyType looks like this:
class __declspec(dllexport) MyType {
public:
union {
struct {
unsigned int Id : 23;
unsigned int Flag : 1;
unsigned int Type : 4;
unsigned int Unused : 4; /* 32 bits total */
};
unsigned int All_Bits; /* Full 32 bits of MyType */
};
/* There are some function definitions here, but no other
variables, aside from some statically defined ones. */
};
UPDATE: This is the more succinct, optimized version of Is_Inst()'s disassembly, which shows the problem. I've removed the old version that was here before for brevity.
// MyType MyType::Is_Inst () {
// uchar Bar=0;
// MyType Foo={0};
mov dword ptr [rdx],0 /* %rdx is 0x17, from a prev fn call. */
// return Foo;
mov rax,rdx
// }
ret
The code leading up to Is_Inst() getting called:
...
for (Counter=0; Counter<N_Items; Counter++) {
...
myOther32BitType = arrayOfMyOtherTypes [Counter]; /* Debugger shows this is ok. */
if (myOther32BitType.8BitField==UNEQUAL_ENUM_VALUE) {
/* Some stuff that doesn't happen. */
}
/* myOther32BitType.8BitField==0x17, so %rdx gets set to 0x17. */
else if (strchr((char*)Uchar_Table_Of_Enum_Values, myOther32BitType.8BitField)) continue;
/* %rdx gets set to 0x17 again. */
else if (strchr((char*)Other_Uchar_Table_Of_Enum_Values, myOther32BitType.8BitField)) continue;
else if ( myOther32BitType.8BitField==EQUAL_ENUM_VALUE) {
if (myType.Is_Inst ().All_Bits) { /* Is_Inst() called here. */
return false;
}
...
}
...
}
I've posted a bug report to MS here: https://connect.microsoft.com/VisualStudio/feedback/details/674672/callee-disassembly-expects-address-which-caller-is-not-providing-in-x64-mode
Again, thanks to all for your help!
This seems like the same bug Qt people encountered in VS2010. The report links to Microsoft Connect here. And the bug should be fixed in SP1. There were numerous optimization b ugs fixed in SP1, so I may not have linked the correct one (found three others just sifting through links).
else if (strchr((char*)Other_Uchar_Table_Of_Enum_Values, myOther32BitType.8BitField)) continue;
This looks horrifically wrong. Instead of resorting to completely unnecessary casting, you should use std::find
.
In fact, your whole code looks littered with C-style unsafe code. Where's the memory management objects?
精彩评论