Transforming a 3D plane using a 4x4 matrix

I have a shape made out of several triangles which is positioned somewhere in world space with scale, rotate, translate. I also have a plane on which I would like to project (orthogonal) the shape.

I could multiply every vertex o开发者_StackOverflow社区f every triangle in the shape with the objects transformation matrix to find out where it is located in world coordinates, and then project this point onto the plane.

But I don't need to draw the projection, and instead I would like to transform the plane with the inverse transformation matrix of the shape, and then project all the vertices onto the (inverse transformed) plane. Since it only requires me to transform the plane once and not every vertex.

My plane has a normal (xyz) and a distance (d). How do I multiply it with a 4x4 transformation matrix so that it turns out ok?

Can you create a vec4 as xyzd and multiply that? Or maybe create a vector xyz1 and then what to do with d?

You need to convert your plane to a different representation. One where N is the normal, and O is any point on the plane. The normal you already know, it's your (xyz). A point on the plane is also easy, it's your normal N times your distance d.

Transform O by the 4x4 matrix in the normal way, this becomes your new O. You will need a Vector4 to multiply with a 4x4 matrix, set the W component to 1 (x, y, z, 1).

Also transform N by the 4x4 matrix, but set the W component to 0 (x, y, z, 0). Setting the W component to 0 means that your normals won't get translated. If your matrix is composed of more that just translating and rotating, then this step isn't so simple. Instead of multiplying by your transformation matrix, you have to multiply by the transpose of the inverse of the matrix i.e. Matrix4.Transpose(Matrix4.Invert(Transform)), there's a good explanation on why here.

You now have a new normal vector N and a new position vector O. However I suppose you want it in xyzd form again? No problem. As before, xyz is your normal N all that's left is to calculate d. d is the distance of the plane from the origin, along the normal vector. Hence, it is simply the dot product of O and N.

There you have it! If you tell me what language you're doing this in, I'd happily type it up in code as well.

EDIT, In pseudocode:

The plane is vector3 xyz and number d, the matrix is a matrix4x4 M

vector4 O = (xyz * d, 1)
vector4 N = (xyz, 0)
O = M * O
N = transpose(invert(M)) * N
xyz = N.xyz
d = dot(O.xyz, N.xyz)

xyz and d represent the new plane

This question is a bit old but I would like to correct the accepted answer.
You do not need to convert your plane representation.

Any point lies on the plane if
It can be written as dot product :

You are looking for the plane transformed by your 4x4 matrix .
For the same reason, you must have

So and with some arrangements

TLDR : if p=(a,b,c,d), p' = transpose(inverse(M))*p

Notation:

• n is a normal represented as a (1x3) row-vector
• n' is the transformed normal of n according to transform matrix T
• (n|d) is a plane represented as a (1x4) row-vector (with n the plane's normal and d the plane's distance to the origin)
• (n'|d') is the transformed plane of (n|d) according to transform matrix T
• T is a (4x4) (affine) column-major transformation matrix (i.e. transforming a column-vector t is defined as t' = T t).

Transforming a normal n:

n' = n adj(T)

Transforming a plane (n|d):

(n'|d') = (n|d) adj(T)

Here, adj is the adjugate of a matrix which is defined as follows in terms of the inverse and determinant of a matrix:

T^-1 = adj(T)/det(T)

Note:

• The adjugate is generally not equal to the inverse of a transformation matrix T. If T includes a reflection, det(T) = -1, reversing the winding order!

• Re-normalizing n' is mathematically not required (but maybe numerically depending on the implementation) since scaling is taken care off by the determinant. Thanks to Adrian Leonhard.

• You can directly transform the plane without first decomposing and recomposing a plane (normal and point).