Which classification algorithm to choose?

I would like to classify text documents into four categories. Also I have lot of samples which are already classified that c开发者_如何学Can be used for training. I would like the algorithm to learn on the fly.. please suggest an optimal algorithm that works for this requirement.

If by "on the fly" you mean online learning (where training and classification can be interleaved), I suggest the k-nearest neighbor algorithm. It's available in Weka and in the package TiMBL.

A perceptron will also be able to do this.

"Optimal" isn't a well-defined term in this context.

there are several algorithms which can be learned on fly. Examples: k-nearest neighbors, naive Bayes, neural networks. You can try how appropriate each of these methods are on a sample corpus.

Since you have unlabeled data you might want to use a model where this helps. The first thing that comes to my mind is nonlinear NCA: Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure, (Salakhutdinov, Hinton).

Well....I have to say that document classification is kind of different what you guys are thinking.

Typically, in document classification, after preprocessing, the test data is always extremely huge, for example, O(N^2)...Therefore it might be too computationally expensive.

The another typical classifier that came into my mind is discriminant classifier...which doesn't need the generative model for your dataset. After training, you have to do is to put your single entry to the algorithm, and it is gonna be classified.

Good luck with this. For example, you can check E. Alpadin's book, Introduction to Machine Learning.