Rectify perspective image of planar structure given the normal vector of the plane and viewing direction of the camera

I have taken images of the facade of a building under multiple view points. Then I use bundler to reconstruct the 3D point cloud of the building. Then I use a RANSAC method to find the plane of the facade in the point cloud.

This gives me the parameters for the plane and its surface normal.

From bundler I also get the camera rotation matrices, which I u开发者_运维百科se to calculate the viewing directions of the cameras.

Now given that I have the normal of the plane and the camera viewing directions, I would like to use this to rectify the images, i.e. as if the images where taken with the camera's viewing direction being the same as the normal of the plane.

How can I calculate the projective transformation matrix from these two vectors?

The plane normal n can be brought in line with the camera's optical axis o with a rotation. The axis of this rotation is found by taking the cross product of -n and o and the amount to rotate is computed from the angle between these vectors (use dot product). This information is sufficient to build up a 3x3 rotation matrix that you can then use to warp the image.

You may find the following paper very useful as it goes into further detail. It also presents some simplifications that reduce the computation to just a few basic arithmetic operations on the components of the plane normal.

Constraints on length and angle KI Kanatani - Computer Vision, Graphics, and Image Processing, 1988 - Elsevier

http://www.iim.cs.tut.ac.jp/~kanatani/papers/cvgip41-88.pdf