how to remove text block (pattern) from a file with sed/awk

I have thousands of text files that I have imported that contain a piece of text that I would like to remove.

It is not just a block of text but a pattern.

<!--
# Translator(s):
#
# username1 <email1>
# us开发者_开发知识库ername2 <email2>
# usernameN <emailN>
#
-->

The block if it appears it will have 1 or more users being listed with their email addresses.

I have another small awk program that accomplish the task in a very few rows of code. It can be used to remove patterns of text from a file. Start as well as stop regexp can be set.

# This block is a range pattern and captures all lines between( and including )
# the start '<!--' to the end '-->' and stores the content in record $0. 
# Record $0 contains every line in the range pattern.
# awk -f remove_email.awk yourfile

# The if statement is not needed to accomplish the task, but may be useful.
# It says - if the range patterns in $0 contains a '@' then it will print
# the string "Found an email..." if uncommented.

# command 'next' will discard the content of the current record and search
# for the next record.
# At the same time the awk program begins from the beginning.


/<!--/, /-->/ {
    #if( $0 ~ /@/ ){
        # print "Found an email and removed that!"
    #}
next
}

# This line prints the body of the file to standard output - if not captured in
# the block above.
1 {
    print
}

Save the code in 'remove_email.awk' and run it by: awk -f remove_email.awk yourfile

This sed solution might work:

 sed '/^<!--/,/^-->/{/^<!--/{h;d};H;/^-->/{x;/^<!--\n# Translator(s):\n#\(\n# [^<]*<email[0-9]\+>\)\+\n#\n-->$/!p};d}' file

An alternative (perhaps better solution?):

sed '/^<!--/{:a;N;/^-->/M!ba;/^<!--\n# Translator(s):\n#\(\n# \w\+ <[^>]\+>\)+\n#\n-->/d}' file

This gathers up the lines that start with <!-- and end with --> then pattern matches on the collection i.e. the second line is # Translator(s): the third line is #, the fourth and perhaps more lines follow # username <email address>, the penultimate line is # and the last line is -->. If a match is made the entire collection is deleted otherwise it is printed as normal.

for this task you need look-ahead, which is normally done with a parser.

Another solution, but not very efficient would be:

sed "s/-->/&\n/;s/<!--/\n&/" file |  awk 'BEGIN {RS = "";FS = "\n"}/username/{print}'

HTH Chris

perl -i.orig -00 -pe 's/<!--\s+#\s*Translator.*?\s-->//gs' file1 file2 file3

Here is my solution, if I understood your problem correctly. Save the following to a file called remove_blocks.awk:

# See the beginning of the block, mark it
/<!--/ {
    state = "block_started" 
}

# At the end of the block, if the block does not contain email, print
# out the whole block.
/^-->/ {
    if (!block_contains_user_email) {
        for (i = 0; i < count; i++) {
            print saved_line[i];
        }
        print
    }

    count = 0
    block_contains_user_email = 0
    state = ""
    next
}

# Encounter a block: save the lines and wait until the end of the block
# to decide if we should print it out
state == "block_started" {
    saved_line[count++] = $0
    if (NF>=3 && $3 ~ /@/) {
        block_contains_user_email = 1
    }
    next
}

# For everything else, print the line
1

Assume that your text file is in data.txt (or many files, for that matter):

awk -f remove_blocks.awk data.txt

The above command will print out everything in the text file, minus the blocks which contain user email.