How to distort the desktop screen

I want to change the shape of the desktop screen, so what are displayed on the desktop will be distorted at the same time. And the user can still operate the PC with the mouse on the distorted desktop(Run the applications, Open the "My Computer" and so on). I think I must get the projection matrix of the screen coordinate at first. Then transform the matrix, and map the desktop buffer image to the distorted mesh. Are there any interfaces which can modify the shape of the desktop screen in OpenGL or DirectX? Would you please give me some tip on it. Thank you very much in advance.

Please refer to the picture from http://oi53.tinypic.com/bhewdx.jpg

BR, Haifeng

Addition1:

I'm sorry! Maybe I didn't express clearly what I want to implement. What I want to implement is to modify the shape of the screen. So we can distort the shapes of all the applications which are run on Windows at the same time. For example that the window of "My Computer" will be distorted with the distortion of the desktop screen. And we can still operate the PC with mouse from the distorted desktop(Click the shortcut to run a program).

Addition2: The projection matrix is just my assume. There isn't any desktop projection matrix by which the desktop surface is projected to the screen. What I want to implement is to change the shape of the desktop, as the same with mapping the desktop to an 3D mesh. But the user can still operate the OS on the distorted desktop(Click the shortcut to run a program, open the ie to surf the internet).

Addi开发者_运维知识库tion3: The shapes of all the programs run on the OS are changed with the distortion of the screen. It's realtime. The user can still operate the OS on the distorted screen as usually.

Maybe we can intercept or override the GPU itself to implement the effect.

I'm investigating GDI, I think I can find some clue for that. The first step is to find how to show the desktop on the screen.

As Window Seven with Aero renders the desktop using Direct3D API, it might be possible to hook (by using detours?) the dwm.exe process and change the way the desktop surface is projected to the screen. As far as I know, there is no DWM API you can use to do that.

OK, first off, the link you posted to uses some complex shape, not a simple quad, so a projection matrix to get that effect won't be enough. You'll need the current desktop image as a texture and map that texture onto a mesh. Note that you still can do this with a simple quad, but it's going to give a more simplistic effect (I still suggest you try this first for debugging your setup).

How to capture the desktop contents:

There are basically two ways (APIs) to do this. The first one is old-school and will only give you a single snapshot (it's very hard to update it carefully so that you still use your desktop even after the effect started). The second one will give you more support for interaction, but doesn't work on Windows XP.

1. Use the Windows GDI functions (possible using GDI+ or .NET, whatever) to capture the desktop contents and save that to a DIB bitmap using BitBlt() -- this is what the "print screen" button on your desktop does, only it puts the image to your clipboard. Then, map that to an OpenGL/DirectX texture and proceed with the rest.
2. Use the new Windows DWM Thumbnail feature to proxy the desktop. There is some documentation on how to use this for compositing effects.

How to use the desktop contents:

This involves a few steps, and the code will vary a lot based on your choice of DirectX or OpenGL.

1. Create a full-screen window with 3D rendering support (i.e. setup OpenGL or DirectX for that window, or use some framework with built-in support for one or both).
2. Capture the desktop contents (method 1: once at startup, method 2: repeatedly in your event loop)
3. Transfer the desktop contents from the bitmap to a texture.
4. Create the surface (quad or mesh) and map the texture.
5. Update your window.

More details:

Implementing this requires good knowledge of:

1. 3D animations using DirectX or OpenGL
2. Advanced UI programming in Windows

You might not have all of the knowledge it takes to implement this, so I recommend that you spend some time studying the requested APIs and planning out the data flow in your application.

Also, this post is deliberately vague. When you need more details about the different steps involved using a specific API, feel free to ask other, more focused, questions on SO!

I have no idea how one could do this in Windows, but creating these effects if fairly easy on X11 with AIGLX compositing. One example for such a compositor is Compiz. So it might be worth to have a look at that one.

I know that you want to do it on Windows, but maybe getting familiar with the neccesary tasks with a system where this already works may be a good idea.

Questions 9.100 and 9.110 over here are as follows

9.100 How can I find the screen coordinates for a given object-space coordinate?

You can use the GLU library gluProject() utility routine if you only need to find it for a few vertices. For a large number of coordinates, it can be more efficient to use the Feedback mechanism.

To use gluProject(), you'll need to provide the ModelView matrix, projection matrix, viewport, and input object space coordinates. Screen space coordinates are returned for X, Y, and Z, with Z being normalized (0 <= Z <= 1).

9.110 How can I find the object-space coordinates for a pixel on the screen?

The GLU library provides the gluUnProject() function for this purpose.

You'll need to read the depth buffer to obtain the input screen coordinate Z value at the X,Y location of interest. This can be coded as follows:

GLdouble z;

glReadPixels (x, y, 1, 1, GL_DEPTH_COMPONENT, GL_DOUBLE, &z);

Note that x and y are OpenGL-centric with (0,0) in the lower-left corner.

You'll need to provide the screen space X, Y, and Z values as input to gluUnProject() with the ModelView matrix, Projection matrix, and viewport that were current at the time the specific pixel of interest was rendered.

This OpenGl tutorial says how to reset the Projection Matrix using the following

glMatrixMode(GL_PROJECTION);        // Select The Projection Matrix
glLoadIdentity();                  // Reset The Projection Matrix

// Calculate The Aspect Ratio Of The Window
gluPerspective(45.0f,(GLfloat)width/(GLfloat)height,0.1f,100.0f);

glMatrixMode(GL_MODELVIEW);     // Select The Modelview Matrix
glLoadIdentity();                 // Reset The Modelview Matrix

No, you don't want to "modify the projection matrix by which the desktop surface is projected to the screen".

The desktop surface is NOT projected to the screen. It's in 2D, so it's a simple memcpy() between two arrays (one of which happens to be on the graphic card)

The basic idea is as follow :

• you get your OS to render the desktop in a texture, instead of a standard framebuffer.
• you clear the screen
• you display some triangles and texture it with the "desktop"

This should be possible on Linux with Compiz, as datenwolf suggested, very hard on Vista, with tibur's idea, and almost impossible on XP.

The screenshot you give is under XP, though. I suspect this is actually a second screen, or even a second computer. The desktop is rendered in the standard way, a screenshot is taken, and rendered on a wobbly object on another screen.