How to debug C# managed/unmanaged marshalling exceptions?

I'm writing a DirectShow filter on C# and I don't want to use any third party library. This task is almost 100% based on the correct prototyping of the COM interfaces like IGraphBuilder, IBaseFilter, IPin, etc. And if the interface is prototyped incorrectly various exceptions at managed/native boundaries are thrown. The problem is to determine the location of the incorrectly prototyped interface method.

Currently I have: An exception of type 'System.NullReferenceException' occurred in DirectShow.dll (this is the name of my managed dll) and wasn't handled before a managed/native boundary

Call stack:

ntdll.dll!NtWaitForSingleObject() + 0xa bytes

KernelBase.dll!WaitForSingleObjectEx() + 0x9c bytes

clr.dll!CLREvent::WaitEx() + 0x20f bytes

clr.dll!CLREvent::WaitEx() + 0x1b8 bytes

clr.dll!CLREvent::WaitEx() + 0x73 bytes

clr.dll!Thread::WaitSuspendEventsHelper() + 0xcf bytes clr.dll!Thread::WaitSuspendEvents() + 0x10 bytes

clr.dll!string "d:\iso_whid\amd64fre\base\ntos\r"...() + 0x35688d bytes

clr.dll!Thread::RareDisablePreemptiveGC() + 0x118 bytes

clr.dll!GCHolderEEInterface<0,0,0>::~GCHolderEEInterface<0,0,0>() + 0x19 bytes clr.dll!Debugger::SendCatchHandlerFound() + 0x150 bytes

clr.dll!string "d:\iso_whid\amd64fre\base\ntos\r"...() + 0x3b9340 bytes

clr.dll!NotifyOfCHFFilterWrapper() + 0x77 bytes

clr.dll!string "d:\iso_whid\amd64fre\base\ntos\r"...() + 0x336941 bytes

msvcr100_clr0400.dll!__C_specific_handler() + 0x97 bytes

ntdll.dll!RtlpExecuteHandlerForException() + 0xd bytes ntdll.dll!RtlDispatchException() + 0x38f bytes ntdll.dll!KiUserExceptionDispatch() + 0x2e bytes

KernelBase.dll!RaiseException() + 0x3d bytes

clr.dll!NakedThrowHelper2() + 0xc bytes

clr.dll!NakedThrowHelper_RspAligned() + 0x3d bytes clr.dll!NakedThrowHelper_FixRsp() + 0x5 bytes

000007ff00179486()

clr.dll!COMToCLRDispatchHelper() + 0x4e bytes

clr.dll!SecurityDeclarative::CheckLinkDemandAgainstAppDomain() - 0x40e bytes

clr.dll!COMToCLRWorkerBody() + 0xd6 bytes

clr.dll!COMToCLRWorkerDebuggerWrapper() + 0x22 bytes

clr.dll!COMToCLRWorker() + 0x201 bytes clr.dll!GenericComCallStub() + 0x57 bytes

[Native to Managed Transition]

quartz.dll!CEnumConnectedPins::CEnumConnectedPins() + 0x4a bytes

quartz.dll!CFilterGraph::FindUpstreamInterface() + 0x150 bytes quartz.dll!CFilterGraph::FindUpstreamInterface() + 0xc1 bytes

quartz.dll!CFilterGraph::FindUpstreamInterface() + 0x171 bytes quartz.dll!CFilterGraph::FindUpstreamInterface() + 0xc1 byt开发者_JAVA技巧es

quartz.dll!CFilterGraph::FindUpstreamInterface() + 0x171 bytes quartz.dll!CFilterGraph::FindUpstreamInterface() + 0xc1 bytes

quartz.dll!CWaveSlave::UpdateSlaveMode() + 0xa7 bytes

quartz.dll!CWaveOutInputPin::RemovePreroll() + 0x95 bytes

quartz.dll!CWaveOutInputPin::Receive() + 0x12f bytes

msmpeg2adec.dll!CBaseOutputPin::Deliver() + 0x22 bytes msmpeg2adec.dll!CIVIAudioFilter::DeliverOutSample() + 0x3da bytes

msmpeg2adec.dll!CIVIAudioCodec::DecodeDDPlus() + 0x556 bytes

msmpeg2adec.dll!CIVIAudioCodec::DecodeAll() + 0x121 bytes

msmpeg2adec.dll!CIVIAudioFilter::Process() + 0xda7 bytes

msmpeg2adec.dll!CIVIAudioFilter::Receive() + 0x16d bytes

msmpeg2adec.dll!CTransformInputPin::Receive() + 0x4c bytes msmpeg2adec.dll!CIVIAudioInPin::Receive() + 0x3f bytes quartz.dll!CBaseOutputPin::Deliver() + 0x22 bytes

quartz.dll!CBaseMSRWorker::TryDeliverSample() + 0x14f bytes

quartz.dll!CBaseMSRWorker::PushLoop() + 0x1da bytes

quartz.dll!CBaseMSRWorker::ThreadProc() + 0x90 bytes

quartz.dll!CAMThread::InitialThreadProc() + 0x1c bytes kernel32.dll!BaseThreadInitThunk() + 0xd bytes ntdll.dll!RtlUserThreadStart() + 0x21 bytes

In other words the pipeline is:

1. Native code function named CEnumConnectedPins()
2. Native to managed transition -> If exception is thrown here then the xxx from step 3 is only known to marshaller but we're in the middle of nowhere.
3. Managed code method named xxx.

So this doesn't get me anywhere and I don't know how to debug that.

This is very ugly to debug, the failure occurs in code you didn't write. Go slower to diagnose this. Write a native test program that obtains the interface pointer you want to test and verify the methods one by one, in v-table order. The bad one will pop out.

Beware that C# doesn't support multiple inheritance. Any COM interface that inherits from another interface that isn't IUnknown or IDispatch requires that you repeat the declarations of the methods in the base interface. Forgetting to do this causes the wrong method to be called. Or a non-existing one since the v-table is too short. NullReference or AccessViolation is then a common outcome.