Matlab - bug with linear discriminant analysis

I run

Y_testing_obtained = classify(X_testing, X_training, Y_training);

and the error I get is

Error using ==> classify at 246
The pooled covariance matrix of TRAINING must be positive definite.

X_training is 1550 x 5 matrix. Can 开发者_StackOverflow中文版you please tell me what this error means, i.e. why is it appearing, and how to work around it? Thanks

Explanation: When you run the function classify without specifying the type of discriminant function (as you did), Matlab uses Linear Discriminant Analysis (LDA). Without going into too much details on LDA, the algorithms needs to calculate the covariance matrix of X_testing in order to solve an optimisation problem, and this matrix has to be positive definite (see Wikipedia: Positive-definite matrix). The underlying assumption is that your data is represented by a multivariate probability distribution, which always has a positive definite covariance matrix unless one or more variables are exact linear combinations of the others.

To solve your problem: It is possible that one of your variables is a linear combination of the others. You can try selecting a sensible subset of your variables, or perform Principal Component Analysis (PCA) on the training data and then classify using the first few principal components. Or, you could specify the type of discriminant function and choose one of the two naive Bayes classifiers, for example:

Y_testing_obtained = classify(X_testing, X_training, Y_training, 'diaglinear');

As a side note, you also need to have more observations (rows) than variables (columns), but in your case this is not the problem as you seem to have 1550 observations and 5 variables.

Finally, you can also have a look at the answers posted to a similar question on the Matlab forum.

Try regularizing the data using cvshrink function in Matlab