matching a multiline make-line variable assignment with a python regexp

I am trying to extract from a multiline make-line variable assignment the multiline value. The following testcase fails to find a match in the input string and I have to confess that I fail to see why. Help on making this sample code print "a \ b" on stdout would be most welcome.

#!/usr/bin/env python                                                                                                 

def test():
    s = r"""                                                                                                          
FOO=a \                                                                                                               
  b                                                                       开发者_Python百科                                            
"""
    import re
    print type(s),s
    regex = re.compile(r'^FOO=(.+)(?<!\\)$', re.M)
    m = regex.search(s)
    print m.group(1)

if __name__ == '__main__':
    test()

re.M means re.MULTILINE, but it doesn't concern the symbolism of dot, it concerns the symbolism of ^ and $

You need to specify re.DOTALL to make the dot able to match even with '\n'

def test():
    s = r"""    

FOO=a \    

  b

  """
    import re
    print repr(s)
    print '---------------------'
    regex = re.compile(r'^FOO=(.+)(?<!\\)$', re.M)
    print regex.search(s).group(1)
    print '---------------------'
    regex = re.compile(r'^FOO=(.+)(?<!\\)$', re.M|re.DOTALL)
    print regex.search(s).group(1)

test()

result

'    \n\nFOO=a \\    \n\n  b\n\n  '
---------------------
a \    
-----
'a \\    '
---------------------
a \    

  b


-----
'a \\    \n\n  b\n\n  '

Your problem is that the . doesn't match a newline character by default. If you enable the Dotall modifier it will work.

regex = re.compile(r'^FOO=(.+)(?<!\\)$', re.M | re.S)

You do so using re.S

Your output will then be

a \

 b

Your pattern does just match the pattern including the linebreaks.

I am not sure what you want to achieve with the multi line modifier re.M. It makes the ^ and the $ match a row start/end. I assume you can remove it.

I am also not sure what you want to achieve with your negative lookbehind (?<!\\), I think you should clarify your expected output. (Do you want to remove the newlines in a \ b?)

I came up with this one:

^FOO=((([^\\]*\\\n)*)[^\n]+)

it assumes there are no whitespaces behind the backslash.

Your sample text has a whole lot of space characters in it, including after the backslash. I assume that's not what you intended, since the point of the backslash is to escape the linefeed that would normally mark the end of the entry.

But backslashes can be used to escape other characters as well, including backslashes. If an value happens to end with a backslash, it will show up as two backslashes in the makefile. The lookbehind in your regex will "see" the second one, and incorrectly treat it as part of a line continuation.

If you're thinking of adding another lookbehind to see if the backslash is escaped, let me stop you now. This has been hashed out many times, and the lookbehind approach can't be made to work. What you want is something like this:

regex = re.compile(r'^FOO=([^\n\\]*(?:\\.[^\n\\]*)*)$', re.M | re.S)

See it in action on ideone

The first [^\n\\]* consumes as many non-linefeed, non-backslash characters as it can, then hands control to the next part. If the end of the string hasn't been reached, it tries to match a backslash followed by any character (including linefeeds, thanks to the re.S modifier) followed my some more "normal" characters. It continues like that in a loop until (assuming the input is valid) it runs into an unescaped linefeed or the end of the input.

Although it's the re.S modifier that lets the dot match newlines, the re.M modifier is needed too; it's what lets ^ match the beginning of a line and $ match the end of a line, as @stema explained.