Creating C# to C++ bridge: Why do I get a AccessViolationException when calling DLL?

I'm exploring the idea of building a bridge between a DLL plugin for a 3rd party app and a C# app. I'm writing both the plugin DLL and the C# application. The plugin will be loaded into the 3rd party app and then I want to use call the plugin from C# to indirectly get data from the 3rd party app.

I am able to successfully call an exported function from the DLL from C#. For example:

C++ DLL:

exte开发者_C百科rn "C" __declspec(dllexport) char * HelloFromDll()
{
    char *result;
    result = "Hello from my DLL";
    return result;
}

C#:

using System.Runtime.InteropServices;

[DllImport(@"MyDll.dll")]
private static extern string HelloFromDll();

I can then call this DLL function from C# and display the string in the UI. However, as soon as I create an export function that calls a function from my 3rd party app, I get an AccessViolationException. For example,

extern "C" __declspec(dllexport) char * GetData()
{
    char *result;
    result = 3rdPartyLibrary::SomeFunction();
    return result;
}

Through some testing, the error seems to occur as soon as I make a call to a 3rd party function. How can I fix this?

This function is very difficult to use in a C program as well. Returning strings from functions is a poorly supported scenario. There's a memory management problem, it isn't clear who owns the string. In most cases the caller is expected to take ownership of the string and free it after using it. That's not going to work out well for your function, the program will crash since you returned a string literal.

The .NET pinvoke marshaller needs to solve this problem as well. With the extra problem that it cannot use the allocator that's used by the C code. It is going to call CoTaskMemFree (the COM allocator). That causes an undiagnosable memory leak on XP, a crash on Vista and Win7.

Just don't write C code like this. Always let the caller pass the buffer for the string. Now there's no guessing who owns the memory. Like this:

extern "C" __declspec(dllexport) void HelloFromDll(char* buffer, int bufferSize)
{
    strcpy_s(result, bufferSize, "Hello from my DLL");
}

With your C# code like this:

[DllImport("foo.dll", CharSet = CharSet.Ansi)]
private static extern void HelloFromDll(StringBuilder buffer, int bufferSize);
...
var sb = new StringBuilder(666);
HelloFromDll(sb, sb.Capacity);
string result = sb.ToString();

From your question it seems that this is the scenario:

ProcessA (3rd party App) --> loads X.DLL --> initializes the plugin --> does other stuff.

ProcessB (Your C# App) --> loads X.DLL --> calls GetData();

Does X.DLL loaded in ProcessA have any mechanism to talk to X.DLL loaded in ProcessB?

if not then this approach is flawed. Your code probbably crashes because "3rdPartyLibrary" class hasn't been initialised in your C# app as it is completely different copy of the DLL.

For you to extract this data you need a query interface defined by X.DLL which can talk across processes, maybe sockets?

Then ProcessB talks to this interface and extracts the data. if using sockets, then your X.DLL would implement both server and client code, where your GetData() would use this mechanism (maybe sockets) and query the data and return it.

So : X.DLL in ProcessA should act like a server. And: X.DLL (or write a Y.DLL) in ProcessB should act like a client and get this information from ProcessA.

btw, if the query is only needed to be done once, just hard code this is in X.DLL and dump to disk, and then explore at your convinience :-)

Generally, a returned char* needs to be returned as an IntPtr:

[DllImport(@"MyDll.dll")]
private static IntPtr HelloFromDll();

Then, you'll need to convert that IntPtr into a string:

string retVal=Marshal.PtrToStringAnsi(HelloFromDll());

Strings are a bit difficult in P/Invoke. My general rule of thumb is:

• Input char* parameter = c# string
• Return char * = IntPtr (use PtrToStringAnsi)
• Output char* parameter = c# StringBuilder - and be sure to pre-allocate it large enough before (ie = new StringBuilder(size)) calling the function.