Texture analysis on irregular region of interest

I have an image which I would like to extract the GLCM texture in an area of interest(AOI). But AOI is a non-rectangular shape.

As an image is always stored as a matrix in Matlab, even if the AOI is an irregular polygonal area the neighboring pixels will also have to be used t开发者_运维知识库o make it a rectangular region. Since all the pixels outside the area of interest are made equal to zero, does this affect the features extracted from texture analysis.

Is it possible to do any kind of image analysis on non-rectangular regions?

Yes, if the pixels outside the area of interest were being used when computing the gray level cooccurrence matrix, then the result would be incorrect -- that is, would not suit your requirements, as border processing is a matter of choice.

Existing software systems offer this feature:

If you use matlab, according to http://www.mathworks.com/help/toolbox/images/ref/graycomatrix.html, you would need to assign to the pixels of the input image which are outside the AOI the value Nan.

In Mathematica, very conveniently the function ImageCooccurrence has an option named Masking which allows to pass any AOI as a binary mask. From http://reference.wolfram.com/mathematica/ref/ImageCooccurrence.html: