P-Invoke Unmarshalling an array of structs allocated in a DLL

I have a requirement to shovel large amounts (theoretically up to a few terabytes, only limited by available memory) of data from a C/C++ based core DLL which manages, filters and allocates the data, to a C# GUI (which only reads the data). It would be optimal if the data are never duplicated but only accessed per reference from the C#. The signature of my DLL's method is currently the following (I can make changes there):

extern "C" {

typedef struct {
    wchar_t* name;
    __int32 t;
    float v;
    bool condition;

    } TestData;

__declspec(dllexport) void fillTestArrayWithAlloc(TestData** td, __int32* size);
};

The array length is not known to C# and can be anything between 1 and millions....

I mapped the structure like this:

   [StructLayout(LayoutKind.Sequential)]
   struct TestData
    {
        [MarshalAs(UnmanagedType.LPWStr)]
        public Str开发者_StackOverflowing name;
        public Int32 t;
        [MarshalAs(UnmanagedType.R4)]
        public float v; 
        public bool condition;

    }

I tried both manual unmarshalling like this

static extern void fillTestArrayWithAlloc(ref IntPtr td, ref int size);

and automatic unmarshalling like this

static extern void fillTestArrayWithAlloc([Out] TestData[] td, ref int size);

But in both cases it performs a memory copy, which is undesired. Is there a way to do it without a data copy?

Best regards Petr

The first version of your function declaration should not cause all the data to be copied: only the pointer and length value will be marshalled.

I think you should then be able to implement an iterator or enumerator which marshals one TestData structure at a time directly from the unmanaged memory pointed to by the IntPtr, using Marshal.PtrToStructure. You will need to do some unsafe pointer arithmetic to move from one instance to another as you iterate.

This approach should avoid wholesale copying of the entire structure. Good luck.