Clone detection algorithm

I'm writing an algorithm that detects clones in source code. E.g. if there is a block like:

for(int i = o; i <5; i++){
doSomething(abc);
}

...and if this block is repeated somewhere else in the source code it will be detected as a clone. The method I am using at the moment is to create hashes for lines/blocks and compare them with hashes of other lines/blocks in the same source to see if there are any matches.

Now, if the same block as above was to be repeated somewhere with only the argument of doSomething different, it would not be detected as a clone even though it would appear very much like a clone to you and me. My algorithm detects exact matches but doesn't detect matching blocks where only the argument is different.

Could anyone suggest any ways of getting around thi开发者_StackOverflow中文版s issue? Thanks!

Here's a super-simple way, which might go too far in erasing information (i.e., might produce too many false positives): replace every identifier that isn't a keyword with some fixed name. So you'd get

for (int DUMMY = DUMMY; DUMMY<5; DUMMY++) {
  DUMMY(DUMMY);
}

(assuming you really meant o rather than 0 in the initialization part of the for-loop).

If you get a huge number of false positives with this, you could then post-process them by, for instance, looking to see what fraction of the DUMMYs actually correspond to the same identifier in both halves of the match, or at least to identifiers that are consistent between the two.

To do much better you'll probably need to parse the code to some extent. That would be a lot more work.

Well if you're going todo something else then you're going to have to parse to code at least a bit. For example you could detect methods and then ignore the method arguments in your hash. Anyway I think it's always true that you need your program to understand the code better than 'just text blocks', and that might get awefuly complicated.