How to execute parallel 'for' loop containing if statements
I've this function for processing some images and converting it to pdf. Question is: how to run the for loop in this function in parallel to process the fil开发者_开发问答es more quickly?
pdf_compile() {
start=`date +%s.%N`
tmpdir=".temp"
if [ ! -d "$tmpdir" ]; then
mkdir $tmpdir
fi
echo "width: $1px"
[[ -z $2 ]] && filter="sc1" || filter="$2"
echo "filter:" $filter
count=$(ls | grep .jpg | wc -l)
dn=1
for f in *.jpg ; do
nm="$tmpdir/$1-$filter-$f"
if [ ! -f "$nm" ]; then
echo "processing $f ($dn/$count)"
if [[ "$filter" =~ ^sc1 ]]; then
convert $f -resize $1x \
-brightness-contrast 21x33 \
-enhance -despeckle -sharpen 2 $nm
elif [[ "$filter" =~ ^sc2 ]]; then
convert $f -resize $1x \
-brightness-contrast 19x45 \
-sharpen 1 -despeckle -despeckle \
-enhance $nm
elif [[ "$filter" =~ ^sc3 ]]; then
convert -resize $1x \
-brightness-contrast 19x35 \
-sharpen 1 -enhance -despeckle \
-gamma 1.05 \
-brightness-contrast -30x90 \
-enhance $nm
fi
ffmpeg -i $nm $nm -y -hide_banner -loglevel error
else
echo "using cached $f ($dn/$count)"
fi
((dn++))
done
n="$(basename $PWD)_$1px.pdf"
echo "\nconverting to pdf"
convert $tmpdir/$1-$filter-*.jpg $n
if [[ "$3" =~ ^y ]]; then
echo "removing temp"
rm $tmpdir -r
fi
end=`date +%s.%N`
runtime=$( echo "$end - $start" | bc -l )
echo "completed in" $runtime "s"
}
I tired this: https://unix.stackexchange.com/a/216475
for f in *.jpg ; do
((i=i%N)); ((i++==0)) && wait
.......
done
But it didn't improve the processing time
Your code is incomplete and has quite a few issues. Try pasting it into https://shellcheck.net for some guidance.
You could put &
at the end of each of the 3 convert
statements inside the if
statements in the for
loop to do them all in the background and then add wait
before the final convert
that creates the PDF.
for ...
if ...
convert ... &
else ...
convert ... &
else ...
convert ... &
done
...
wait # for all JPEGs to finish
convert ... XXX.PDF
Rather than create intermediate JPEGs on disk, you could maybe create "Magick Pixel Cache" files by changing the extension to .mpc
and see if that is faster. You could also write them to a RAM-based filesystem like /tmp
instead of the local, presumably disk-based filesystem.
You give no indication of the image sizes. If you start with multi-thousand pixel wide images and go to several hundred pixels width, you could usefully use "shrink-on-load" feature.
You give little indication of how many pages your PDFs have, nor how long each part of the processing takes. Better specified questions are often easier to answer well...
精彩评论