PInvoke in 64bit .net app through c++ 64 bit dll

I'm having an issue calling a function in a c++ dll inside of a c# app. I'm calling the function inside of c# like so:

[DllImport("cryptopp.dll")]
public static extern IntPtr RSAEncryptString(string filename, string seed, string message);

It is being exported in the c++ dll as shown below.

extern "C" __declspec(dllexpo开发者_高级运维rt) const char* __cdecl RSAEncryptString(const char *pubFilename, const char *seed, const char *message);

What I get when I try to call this, however, is an "An External component has thrown an exception." exception, which is not very descriptive at all, and extremely unhelpful.

When I pull up the dll in an export viewer, it shows all the other exported functions with fully quantified declarations (I.E. public: void __cdecl CryptoPP::X509PublicKey::`vbase destructor'(void) __ptr64 ) , except for the function I am calling, which just displays the function name RSAEncryptString.

This is the only possible issue I can see, besides maybe mis-calling the function with an invalid declaration on the c# side. Am I using System.Runtime.InteropServices.Marshal wrong?

Please help <3 and thanks in advance.

I think you need to change the first line to:

[DllImport("cryptopp.dll",
    CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]

If you want to get very descriptive, you can also add these:

public static extern IntPtr RSAEncryptString(
    [In, MarshalAs(UnmanagedType.LPStr)] string filename,
    [In, MarshalAs(UnmanagedType.LPStr)] string seed,
    [In, MarshalAs(UnmanagedType.LPStr)] string message);

IIRC think the CharSet should take care of the encoding thing for you, but if it doesn't, use the MarshalAs also, as shown above.

Edit:

Oh I think I got why you still get an error! Your code still had the above problems, but it's still erring because you can't return a string object since it's not a managed object; you need to return a pointer (like IntPtr) and then use Marshal.PtrToStringAnsi!

(I didn't really look at your return type when answering this at first.)

It appears you're trying to store the return value of type const char * (an LPCSTR) into an IntPtr type (usually used for HANDLEs, not LPSTRs.) Try this:

[DllImport("cryptopp.dll", CharSet = CharSet.Auto)]
public static extern String RSAEncryptString(String filename, String seed, String message);

Also keep in mind that if any argument is getting written to, you'll need to add out before its type, i.e. ..., out String message)