Ruby C Extension - 64bit pointer being truncated causing a segfault
I'm working on trying to get a ruby extension written in c, working with ruby 1.9.2 on Mac OSX. I keep getting a segfault caused by a 64-bit pointer returned from a function being truncated to a 32-bit value and trying to access low memory.
Here's what I've figured out about it. VALUE is a long, defined in the ruby source code, and the code won't compile unless sizeof(long) is 8 bytes.
In file_b.c:
VALUE
rb_helper_function_foo(VALUE file)
{
VALUE blah;
...
//Using gdb, blah right here is a 64bit address, ex: 0x1 009d 62a0
return blah;
}
In file_a.c
VALUE
func_do_stuff(int argc, VALUE *argv, VALUE obj)
{
VALUE filename, file_path;
...
// But here, the size of what is returned is 4 bytes
// file_path has a size of 8 bytes though
file_path = rb_helper_function_foo(filename);
//This statement will segfault :-(
if(!RTEST(file_path))
{
...
}
...
}
At the end of the function call, the return value has the correct address, but only the lower 4 bytes get returned: 0x009d52a0 instead of 0x1009d52a0. Accessing the low memory address is causing a segfault.
In the assembly code that gets called during the return, there's this instruction
movslq %eax, %r12
which only copies the lower 4 byte开发者_高级运维s, but I need all 8 bytes of %rax to be copied.
I'm on ruby 1.9.2-p290 (though it did the same with p180), Mac OSX 10.6.8, using xcode 4.0.1 (with gcc 4.2.1), but I also tried upgrading to gcc 4.6.1. It has also previously tried xcode 3.2.
Thanks for any help you can give me.
Maybe this is as simple as rb_helper_function_foo
not being declared in the file_a.c compilation unit-- so the compiler is assuming it returns int instead of VALUE?
精彩评论