IPreviewHandler throws uncatchable exception

I've imported the COM interface IPreviewHandler into a WinForms app and am using it to display previews for various types of documents (I look up the GUID of the appropriate preview handler in the registry, then use Activator.CreateInstance(guid) to instantiate the specific COM class.

This works wonderfully for the vast majority of file types - Office formats, PDFs, videos, etc - however, after I instantiate the "Microsoft Windows TXT Preview Handler" {1531d583-8375-4d3f-b5fb-d23bbd169f22}, initialise it with a stream containing an ordinary .txt file, set the bounds of the preview window and then finally call DoPreview(), I get an exception that cannot be caught using try...catch:

try {开发者_开发知识库
    Type comType = Type.GetTypeFromCLSID(guid);
    object handler = Activator.CreateInstance(comType);

    if (handler is IInitializeWithStream) {
        Stream s = File.Open(filename, FileMode.Open);
        // this just passes the System.IO.Stream as the COM type IStream
        ((IInitializeWithStream)handler).Initialize(new StreamWrapper(s), 0);
    }
    else {
        throw new NotSupportedException();
    }

    RECT r = new RECT();
    r.Top = 0;
    r.Left = 0;
    r.Right = hostControl.Width;
    r.Bottom = hostControl.Height;

    ((IPreviewHandler)handler).SetWindow(hostControl.Handle, ref r);
    ((IPreviewHandler)handler).DoPreview();    // <-- crash occurs here
}
catch (Exception) {
    // this will never execute
}

When I step through with the debugger, the Visual Studio Hosting Process crashes. With no debugger, the application crashes without firing the AppDomain.UnHandledException or Application.ThreadException events.

I don't really mind that I can't preview plain text files using this technique (the working preview handlers for Office formats, etc are sufficient for my app's requirements), but I am concerned about having my app crash uncontrollably should the user select a .txt file. Is there any way I can catch this error and handle it gracefully? Better yet, is there some way I can overcome it and get the handler to work?

I couldn't get the GetPreviewHandlerGUID() to recognize a .txt file and had to inject the GUID directly. You can see what goes wrong when you use Project + Properties, Debug, tick Enable unmanaged code debugging.

The debugger will now stop on the problem and display

`STATUS_STACK_BUFFER_OVERRUN encountered

The top of the call stack looks like this:

kernel32.dll!_UnhandledExceptionFilter@4()  + 0x1a368 bytes 
shell32.dll!___report_gsfailure()  + 0xc8 bytes 
shell32.dll!CRTFPreviewHandler::_StreamInCallback()  + 0x74 bytes   
msftedit.dll!CLightDTEngine::ReadPlainText()  + 0xed bytes  
msftedit.dll!CLightDTEngine::LoadFromEs()  + 0x202b3 bytes  
msftedit.dll!CTxtEdit::TxSendMessage()  + 0x1e25f bytes 
msftedit.dll!_RichEditWndProc@16()  + 0x13d bytes   

The problem is located in the StreamInCallback() function. It is called by the RichTextBox that's used to display the preview (msftedit.dll) to load the file. The code in this callback function has a bug, it destroys the 'canary' that's used to detect that the stack frame got corrupted because of a buffer overrun.

This is part of the counter-measures that Microsoft took to prevent viruses from injecting themselves by buffer overruns. The /GS compile option in Visual Studio for the C/C++ languages. Once detected, the CRT very swiftly terminates the program. This happens without an exception getting raised, the stack cannot safely be unwound because it was compromised. Accordingly, the CLR cannot catch the exception.

This bug is specific to the TXT file viewer. There's nothing you can do about it other than not using it. Reporting this bug to connect.microsoft.com probably isn't useful, they'll close it as 'external'. This is otherwise a subtle hint what can happen when you let unmanaged code run inside your program ;)

The real reason you're getting this problem is that you're creating the preview handler object in-process. The correct way is to create it out-of-process.

DISCLOSURE The following contains advertisement of my blog/code snippet.

See https://github.com/GeeLaw/PreviewHost for an example. Specifically, see Line 219 of PreviewHandler.cs, where you have to pass CLSCTX_LOCAL_SERVER to CoCreateInstance. As explained in one of my blog entries, Activator.CreateInstance allows in-process server, which does not meet the expectation of preview handlers, since they must be created in the correct surrogate process, as documented on MSDN.

I had the same problem and I was able to get the TXT PreviewHandler working by compiling in x64 instead of AnyCPU.

I am using Visual Studio 2010 on Windows 7 (64bit) so this answer won't apply if you are in a 32bit OS.

In Visual Studio 2010

• click on the Configurations drop-down list
• select Configuration Manager...
• click in the Platform cell next to your project
• select New... and select target platform x64
• copy settings from AnyCPU and away you go.

Its very unlikely but can be the issue here - catch(Exception) will catch only exceptions of type of Exception - try using catch w/o any type filtering.

catch(Exception ex) {
   // Normal logging etc
}
catch
{
   // Exception of types other than System.Exception.
}

I think I've found the solution to this problem. The thing is that the stream you're creating is either cleaned by the garbage collector or something else. If you call the initialize method using the stream created by the code below it should work:

System.Runtime.InteropServices.ComTypes.IStream stream;
    byte[] fileData = System.IO.File.ReadAllBytes(filename);
    System.IntPtr hGlobal = System.Runtime.InteropServices.Marshal.AllocHGlobal(fileData.Length);
    System.Runtime.InteropServices.Marshal.Copy(fileData, 0, hGlobal, fileData.Length);
    NativeMethods.CreateStreamOnHGlobal(hGlobal, false, out stream);
    //[DllImport("ole32.dll")]
    //internal static extern int CreateStreamOnHGlobal(IntPtr hGlobal, bool fDeleteOnRelease, out IStream ppstm);

I’m using the code above in a Windows Forms Application explicitly set to 32bit (x86) and running in Single-Threaded Apartments mode.

Credit goes to Sherlock Homes (http://www.tech-archive.net/Archive/DotNet/microsoft.public.dotnet.framework.interop/2010-09/msg00003.html)