C# interop AccessViolationException with managed callback

I have an unmanaged dll which allocates a struct and passes out a pointer to that struct. I have created a c# equivalent of that struct and can happily compile and run my code to utilise it. The struct has an optional pointer within it to allow you to hook up a function pointer that will be called when the unmanaged code runs. When I try to hook a managed delegate to the struct's pointer and pass it back in it blows up with an AccessViolationException. What am I missing?

Some more detail:

unmanaged c code:

typedef struct MyStruct {
   :
   :
   int flags
   :
   int (*cback)(MyStruct *s, Other *o)
   :
} MyStruct;

C# equivalent:

[StructLayout(LayoutKind.Sequential)]
public class MyStruct
{
   :开发者_如何学编程
   :
   [MarshalAs(UnmanagedType.I4)]
   public int flags;
   :
   public IntPtr cback;
   :
};

Having got a pointer to the unmanaged structure I

managedMyStruct = (MyStruct)
    Marshal.PtrToStructure(pUnmanagedMyStruct, typeof(MyStruct));                    

managedMyStruct.cback = 
    Marshal.GetFunctionPointerForDelegate(ManagedDelegateRef);                    

// Update pointer
Marshal.StructureToPtr(managedMyStruct, pUnmanagedStruct, true);

When I pass pUnmanagedStruct in to the unmanaged function that eventually calls the cback, my cback delegate gets called once and the application blows up with an AccessViolationException.

Any clues gratefully received.

A

What is ManagedDelegateRef pointing to? A static method or an instance method? If it's an instance method, make sure the the instance doesn't get garbage collected.

I ran into something similar: I passed a pointer to a managed callback to unmanaged code and when the callback got called the function ran once, then the program crashed.

I didn't get an AccessViolationException -- I didn't get any exceptions -- but maybe the cause of your problem is the same as mine.

The fix for my problem was as follows:

According to [1] there are different function calling conventions: the older __cdecl and the newer __stdcall; unmanaged C/C++ use __cdecl by default and C# uses __stdcall by default.

I'm guessing that your unmanaged code is using the default __cdecl convention. If you can change the convention in your unmanaged code then that might be your fix.

Unfortunately for me I was using a third party DLL and couldn't change the unmanaged calling convention in it. What needed to be done for my program was to tell C# that the delegate I was passing was to use the __cdecl convention.

Unfortunately there is no way to tell this to C# directly. (You'd think there'd be an attribute that could be used but apparently MS hasn't implemented one for C#, though I believe managed C++ has one).

In order to get around this a bit of a hack needed to be used:

The output of the program (DLL/EXE) needed to be decompiled using the ildasm command in the Visual Studio Command Prompt:

cmd>  ildasm /out=output.il OUTPUT.EXE

An attribute was then added to the Invoke method of the delegate in the IL code to tell it to use the __cdecl calling convention:

// output.il
.
.
.

.class public auto ansi sealed NAMESPACE.ManagedDelegate
       extends [mscorlib]System.MulticastDelegate
{
  .custom instance void NAMESPACE.UseCCallingConventionAttribute::.ctor() = ( 01 00 00 00 ) 
  .method public hidebysig specialname rtspecialname 
          instance void  .ctor(object 'object',
                               native int 'method') runtime managed
  {
  } // end of method ManagedDelegate::.ctor

  .method public hidebysig newslot virtual 
          instance void  Invoke(native int pUser,
                                int32 state) runtime managed
  {
  } // end of method ManagedDelegate::Invoke

  .method public hidebysig newslot virtual 
          instance class [mscorlib]System.IAsyncResult 
          BeginInvoke(native int pUser,

.
.
.

became:

.
.
.

.class public auto ansi sealed NAMESPACE.ManagedDelegate
       extends [mscorlib]System.MulticastDelegate
{
  .custom instance void NAMESPACE.UseCCallingConventionAttribute::.ctor() = ( 01 00 00 00 ) 
  .method public hidebysig specialname rtspecialname 
          instance void  .ctor(object 'object',
                               native int 'method') runtime managed
  {
  } // end of method ManagedDelegate::.ctor

  .method public hidebysig newslot virtual 
          instance void #####modopt([mscorlib]System.Runtime.CompilerServices.CallConvCdecl)##### Invoke(native int pUser,
                                int32 state) runtime managed
  {
  } // end of method ManagedDelegate::Invoke

  .method public hidebysig newslot virtual 
          instance class [mscorlib]System.IAsyncResult 
          BeginInvoke(native int pUser,

.
.
.

sans the hashes. (The name of the delegate type in this example is ManagedDelegate -- I wasn't sure what your type name was.)

(Note: [1] and [2] recommend putting a placeholder attribute on the delegate so you can easily find the method in the .il file; UseCCallingConventionAttribute was mine.)

Then the code file was re-compiled with ilasm:

cmd>  ilasm output.il

with /DLL or /EXE, depending on your output type -- /EXE is default.

And this was the fix that worked for my project.

This whole process is outlined in a bit more detail in [1], and someone posted a Perl script to do it in [2]. I have yet to automate things and am a VS n00b so I don't know if this can be added as a step in the build or not, but there it is.

Hope this helps.

[1] http://www.codeproject.com/KB/cs/cdeclcallback.aspx

[2] http://www.dotnet247.com/247reference/msgs/17/87210.aspx