Scikits-learn: Use custom vocabulary together with Pipeline
In my scikits-learn Pipeline, I would like to pass a custom vocabulary to CountVectorizer():
text_classifier = Pipeline([
('count', CountVectorizer(vocabulary=myvocab)),
('tfidf', TfidfTransformer()),
('clf', LinearSVC(C=1000))
])
However, as far as I 开发者_如何学Gounderstand when I call
text_classifier.fit(X_train, y_train)
Pipeline uses the fit_transform() method of CountVectorizer(), which ignores myvocab. How could I modify my Pipeline to use myvocab? Thanks!
This was a bug in scikit-learn that I fixed five minutes ago. Thanks for spotting it. I suggest you either upgrade to the newest version from Github, or separate the vectorizer from the pipeline as a workaround:
count = CountVectorizer(vocabulary=myvocab)
X_vectorized = count.transform(X_train)
text_classifier = Pipeline([
('tfidf', TfidfTransformer()),
('clf', LinearSVC(C=1000))
])
text_classifier.fit(X_vectorized, y_train)
UPDATE: since this answer was posted, this fix has been incorporated in several scikit-learn releases.
精彩评论