Combining two files in different folders in Linux

I have two set of folders that have files with the same filenames and structure. The folder structure is something like this:

\outputfolder\
 |---\folder1\
 |      |---file1.txt
 |      |---file2.txt
 |
 |---\folder2\
        |---file1.txt
        |---file2.txt

So what I need to do is to combine (append) all the files with the same name in these folders (file1.txt开发者_C百科 with file1.txt etc.) into another file inside the outputfolder. After getting these combined files I also need to create a tar.gz file from all of these combined files.

How can I accomplish this in a Linux based command line environment? The folder name (folder1 and folder2 etc) is variable so this needs to be given but the files need not and it should automatically combine all the files with the same name.

Also, these files have headers for column names, so I would need to remove that as well while appending.

Here's some code to get you started

topdir=outputfolder
dir1=folder1
dir2=folder2

for f in $topdir/$dir1/*.txt
do
    outf=$topdir/`basename $f .txt`-concat.txt
    cp $f $outf
    sed -e '1 d' $topdir/$dir2/`basename $f` >> $outf
done

tar czf foo.tar.gz $topdir/*-concat.txt

Edit: added the part removing the header of the 2nd file.

find . -name 'file1.txt' | xargs cat >file1_concat.txt

This will work even if there are some files only in folder1 and some files only in folder2:

concat_files() {
  for dir in "$@"; do
    for file in "$dir"/*; do 
      this=$(basename "$file")
      { [[ -f "$this" ]] && sed 1d "$file" || cat "$file"; } >> "$this"
    done
  done
  tar zcvf allfiles.tar.gz *
}

concat_files folder1 folder2

It will work if you have more than 2 folders for your concatenation job.

I assume you want to keep the header in the resulting file.

Have you tried the cat command (concatenation)?

cat file1 file2 >> outputfile

Might want to chuck this in a small bash script to go through the directory. This should start you off.

Best of luck. Leo