开发者

Algorithm to compare similarity of English sentences

I have开发者_开发百科 a collection of sentences, and I need to analyse them to see how similar they are.

Are there any established algorithms to do this?

I care about:

  • containing the same words (ignoring inflexions for now)
  • containing the same words in a similar order

I've used Levenshtein distance and n-grams for spelling before, although I'm not entirely confident if these translate to my purposes.

Naively, "I don't care about spelling differences, typos can be treated as different words" although perhaps it would be nice to account for this.

perhaps some hybrid of splitting the sentence at spaces and one of the above (or other) algorithms would be a starting point

What options are available? Any advice?

Thanks!


This paper compares several sentence similarity measures. Perhaps you can use one of them as is, or modify it for your needs.

Otherwise sentence similarity measure is a good key term to google for.


To ignore inflections you should look into stemming algorithms: http://en.wikipedia.org/wiki/Porter_stemmer

They reduce words to their root forms.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜