开发者

efficient method to replace multiple words in text

Using JavaScript I need to efficiently remove ~10000 keywords from a ~100000 word document, of which ~1000 will be keywords. What approach would you suggest?

Would a massive regular expression be practical? Or should I just iterate throug开发者_高级运维h the document characters looking for keywords (boring)?

Edit:

Good point - only whole words, not parts. And some keywords contain spaces.

I am trying to do it all client side to reduce pressure on the backend.


Using a regular expression might be a good option:

var words = ['bon', 'mad'];
'joe bon joe mad'.replace(new RegExp('(' + words.join('|') + ')', 'g'), '');
// 'joe  joe  '

The regex1 isn't very complicated with things like look-ahead, and the regexp engine is written in C/C++, so you can expect it be quite fast. Nevertheless - benchmark and see if the performance fits your needs.

I don't think that implementing your own parser will be faster, but I might be wrong - benchmark.

Sending the document to the server doesn't sound very good to me. With 100k words you are looking at a payload in the megabytes range, and you still have to do something with it on the server and push it back.


1 You might have to tune the regexp to do something with the spaces.


My instinct tells me that for such a large number of keywords - sorting the keywords and creating a per character state machine would be much faster than a regular expression, since the state machine is trivial, it can be generated automatically.


A state machine seems to be often used for similar tasks, e.g. http://www.codeproject.com/KB/string/civstringset.aspx

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜