Regex which ignores comments

being a regex beginner, I need some help writing a regex. It should match a particular pattern, lets say "ABC". But the pattern shouldn't be matched when it is used in comment (' being the comment sign). So XYZ ' ABC shouldn't match. x("teststring ABC") also shouldn't match. But ABC("teststring ' xxx") has to match to end, that is xxx not being cut off. Also does anybody know a free Regex application that you can us开发者_如何学Ce to "debug" your regex? I often have problems recognizing whats wrong with my tries. Thanks!

Some will swear by RegexBuddy. I've never used the debugger, but I advise you to steer away from the regex generator it provides. It's just a bad idea.

You may be able to pull this off with whatever regex flavor you're using, but in general I think you're going to find it easier and more maintainable to do this the "hard" way. Regular expressions are for regular languages, and nested anything usually means that regexes aren't a good idea. Modern extensions to regex syntax means it may be doable, but it's not going to be pretty, and you sure won't remember what happened in the morning. And one place where regular expressions fail quite spectacularly (even with modern non-regular extensions) is parsing nested structures - trying to parse any mixture comments, quoted strings, and parenthesis quickly devolves into an incomprehensible and unmaintainable mess. Don't get me wrong - I'm a fan of regular expressions in the right places. This isn't one of them.

On the topic of good regex tools, I really like RegexBuddy, but it's not free.

Other than that, a regex is the wrong tool for the job if you need to check inside string delimiters and all sorts too. You need a finite-state machine.

Odd that lots of people recommend their favorite tools, but nobody provides a solution for the problem at hand. (I'm the developer of RegexBuddy, so I'll refrain from recommending any tools.)

There's no good way of matching Y unless it's part of XYZ with a single regular expression. What you can do is write a regex that matches both Y and XYZ: Y|XYZ. Then use a bit of extra code to process the matches for Y, and ignore those for XYZ. One way to do that is with a capturing group: (Y)|XYZ. Now you can process the matches of the first capturing group. When XYZ matches, the capturing group doesn't match anything.

To do this for your VB-style comments, you can use the regex:

'.*|(ABC)

This regex matches a single quote and everything up to the end of the line, or ABC. This regex will match all comments (whether those include ABC or not). The capturing group will match all occurrences of ABC, except those in comments.

If you want your regex to both skip comments and strings, you can add strings to your regex:

'.*|"[^"\r\n]*"|(ABC)

I find the best 'debugger' for regexes is just messing around in an interactive environment trying lots of small bits out. For Python, ipython is great; for Ruby, irb, for command-line type stuff, sed...

Just try out little pieces at a time, make sure you understand them, then add an extra little bit. Rinse and repeat.

For NET development you might as well try RegexDesigner, this tool can generate code(VB/C#) for you. It is a very good tool for us Regex starters.

link text

Here is my solution to this problem: 1. Find a store all your comments in hash 2. Do your regexp replacement 3. Bring comments back to file

Save your time :-)

string fileTextWithComments = "Some tetx file contents";

Dictionary<string, string> comments = new Dictionary<string, string>();

// 1. Find a store all your comments in hash
Regex rc = new Regex("(?:/\\*(?:[^*]|(?:\\*+[^*/]))*\\*+/)|(?://.*)");
MatchCollection matches = rc.Matches(fileTextWithComments);

int index = 0;
foreach (Match match in matches)
{
    string key = string.Format("/*Comment#{0}*/", index++);
    comments.Add(key, match.Value);
    fileTextWithComments = fileTextWithComments.Replace(match.Value, key);
}

// 2. Do your regexp replacement
Regex r = new Regex("YOUR REGEXP PATTERN");
fileTextWithComments = r.Replace(fileTextWithComments, "NEW STRING");


// 3. Bring comments back to file :-)
foreach (string key in comments.Keys)
{
    string comment = comments[key];
    fileTextWithComments = fileTextWithComments.Replace(key, comment);
}

Could you clarify? I read it thrice, and I think you want to match a given pattern when it appears as a literal. As in not as part of a comment or a string.

What your asking for is pretty tricky to do as a single regexp. Because you want to skip strings. Multiple strings in one line would complicate matters.

I wouldn't even try to do it in one regexp. Instead, I'd pass each line through a filter first, to remove strings, and then comments in that order. And then try and match your pattern.

In Perl because of it's regexp processing power. Assuming @lines is a list of lines you want to match, and $pattern is the pattern you want to match.

@matches =[];
for (@lines){
  $line = $_;
  $line ~= s/"[^"]*?(?<!\)"//g;
  $line ~= s/'.*//g;
  push @matches, $_ if $line ~= m/$pattern/;
}

The first substitution finds any pattern that starts with a double quotation mark and ends with the first unescaped double quote. Using the standard escape character of a backspace. The next strips comments. If the pattern still matches, it adds that line to the list of matches.

It's not perfect because it can't tell the difference between "a\\" and "a\" The first is usually a valid string, the later is not. Either way these substitutions will continue to look for another ", if one isn't found the string isn't thrown out. We could use another substitution to replace all double backslashes with something else. But this will cause problems if the pattern you're looking for contains a backslash.

You can use a zero width look-behind assertion if you only have single line comments, but if you're using multi-line comments, it gets a little trickier.

Ultimately, you really need to solve this kind of issue with some sort of parser, given that the definition of a comment is really driven by a grammar.

This answer to a different but related question looks good too...

If you have Emacs, there is a built-in regex tool called "regexp-builder". I don't really understand the specifics of your regex question well enough to suggest an answer to that.

RegEx1: (-user ")(.*?)"

Subject: report -user "test user" -date 1/4/13 -day monday -daterange "1/4/13 1/20/13" -

Result: -user "test user"

Regex2: (-daterange ")(.*?)"

Subject: report -user "test user" -date 1/4/13 -day monday -daterange "1/4/13 1/20/13" -

Result: -daterange "1/4/13 1/20/13"

RegEx3: (-date )(.*?)( -)

Subject: report -user "test user" -date 1/4/13 -day monday -daterange "1/4/13 1/20/13" -

Result: -date 1/4/13 -

RegEx4: (-day )(.*?)( -)

Subject: report -user "test user" -date 1/4/13 -day monday -daterange "1/4/13 1/20/13" -

Result: -day monday -

Search for the quoted value first if not found, search for the no quotes parameter. This expects only one occurrence of the parameter. It also expects the command to either; use quotes to encapsulate a string with no quotes inside, or; use any character other than a quote in the first position, have no occurrence of ' -' until the next parameter, and have a trailing ' -' (add it onto the string before the regex).