bash - Output to the same file sequence -
suppose have myscript.sh below:
#!/bin/bash $1 > bla.txt bla.txt > temp.txt ... cat temp.txt >> finalouput.txt then run parallel below:
parallel myscript.sh {} ::: {1..3} does write output in order? finaloutput.txt have results of 1 first, 2, , 3.
note: outputting separate files merging them in required order once parallel complete, wondering if avoid step.
the processes run in parallel. not there no guarantee finish in order, there's not guarantee can have multiple processes writing same file , end useful.
if going writing same file multiple processes, should implement sort of locking prevent corruption. example:
while ! mkdir finaloutput.lock; sleep 1 done cat temp.txt >> finaloutput.txt rmdir finaloutput.lock if order matters, should each script write unique file, , assemble final output in correct order after parallel jobs have finished.
#!/bin/bash $1 > bla.txt bla.txt > temp-$1.txt ... cat temp.txt >> finalouput.txt and after parallel has finished:
cat temp-*.txt > finaloutput.txt
Comments
Post a Comment