Create a dataset: extract features from text documents (TF-IDF)
I've to create a dataset from some text files, writing them as vectors of features.
Something like this:
doc1: 1,0.45 6,0.001 94,0.1 ...
doc2: 3,0.5 98,0.2 ...
...
each position of the vector represent a word, and the score is 开发者_如何学JAVAgiven by something like TF-IDF.
Do you know some library/tool/whatever for this? (java is better)
After some days i found the "perfect tool" for this: Word Vector Tool. http://sourceforge.net/projects/wvtool/
mallet. including TF-IDF, POS, classification.
Sure there are many eg http://en.wikipedia.org/wiki/Lucene
However
I recommend that you write an basic IR system from scratch. Looking under the hood is always a great learning experience.
精彩评论