WPF - simulate mouse events on off-screen rendered Grid

I'm rendering a WPF grid with multiple elements (buttons, textbox, ...) to a bitmap which is then used as a texture for a 3D surface in a Direct3D scene. For user interaction I create a 3D ray from the 2D mouse cursor position into the 3D scene finding the intersection point with the gui surface. So I know where the user has clicked on the WPF grid, but from there I'm stuck:

How can I simulate mouse events on the WPF elements while they are not actually shown in an open window but rendered off-screen?

Recently, I was looking into UIAutomation and RaiseEvent but these are used to send events to individual elements, not the whole visual tree. Traversing the tree manually and looking for elements at the cursor position would be an option but I haven't found a way to do this accurately. VisualTreeHelper.HitTest is a good start but instead of finding TextBox it finds TextBoxView and instead of ListBox it finds a Border.

EDIT: returning HitTestResultBehavior.Continue in the result callback of HitTest lets me walk through all elements at a given point. I can now send mouse events to all these elements but the values of the MouseEventArgs object are those of the real mouse. So I have to create a custom MouseDevice which apparently is impossible.

PointHitTestParameters p = new PointHitTestParameters(new Point(
    ((Vector2)hit).X * sourceElement.ActualWidth, 
    (1 - ((Vector2)hit).Y) * sourceElement.ActualHeight));
VisualTreeHelper.HitTest(sourceElement,
     new HitTestFilterCallback(delegate(DependencyObject o)
     {
         return HitTestFilterBehavior.Continue;
     }),
     new HitTestResultCallback(delegate(HitTestResult r)
   开发者_运维问答  {
         UIElement el = r.VisualHit as UIElement;
         if (el != null)
         {
             MouseButtonEventArgs e = new MouseButtonEventArgs(Mouse.PrimaryDevice, 0, MouseButton.Left);
             if (leftMouseDown) e.RoutedEvent = Mouse.MouseDownEvent;
             else e.RoutedEvent = Mouse.MouseUpEvent;
             el.RaiseEvent(e);
         }
         return HitTestResultBehavior.Continue;
     }), p);

You might be able to send windows messages (like WM_MOUSEMOVE) to the WPF window's HWND, via the PostMessage(..) win32 method. I believe these messages would be read by WPF and executed as if it came from a user.

If you are feeling really brave, you can try my hack to get access to the WPF IDirect3DDevice9Ex. From there you can copy the backbuffer in the swapshchain to your d3d scene. If you are using 9Ex (Vista D3D9), you can take advantage of the shared resources (share surfaces/textures/etc between devices and processes) feature to help with performance.

You may still have to do some trickery with windows messages for interactivity.