Shell Programming and Scripting

What output file is (or output files are) supposed to be created?

It looks like you want to run your awk script 250 times each with a pair of input files, but the output from each of those 250 runs goes to a single output file and that single output file is overwritten (not appended to) each time your awk script is run.

Is the intent to create 250 different output files with a name corresponding to the names of the input files, or do you want one output file containing the concatenated contents of the 250 individual file comparisons? If it is one output file that you want, does there need to be some header to each section of the output specifying the input files processed to produce the following section of data in that output file? And, if so, what is the format of that header?

The awk does run on pairs and after running the awk is to create 250 different output files with a name corresponding to the names of the input files. Thank you .

And what are the names of those output files supposed to be???

If the below are the files being used in the awk, comparing F13_ref_FP_10bp.txt to F13_epilepsy.vcf then the output should be F13_epilepsy_comparison.txt.

H19_ref_FP_10bp.txt to H19_marfan.vcf then the output should be H19_marfan_comparison.txt. The input into the awk and the output is tab delimited. Thank you .

REF

Code:

F13_ref_FP_10bp.txt
H19_ref_FP_10bp.txt

VAL

Code:

F13_epilepsy.vcf
H19_marfan.vcf

If we go back to post #3 in this thread, you said that you have a file named out containing pairs of input filenames:

Code:

F13_ref_FP_10bp.txt F13_epilepsy.vcf
M29_ref_FP_10bp.txt M29_epilepsy
H68_ref_FP_10bp.txt H68_marfan.vcf
T42_ref_FP_10bp.txt T42_epilepsy.vcf
H19_ref_FP_10bp.txt H19_marfan.vcf
T48_ref_FP_10bp.txt T48_marfan.vcf

and from post #1 we have pairs of filenames:

Code:

S1234_ref.txt S1234_panel.vcf
A5678_ref.txt A5678_panel.vcf
T1111_ref.txt T1111_panel.vcf

From these examples am I correct in assuming that we can skip creation of the out file and just look at /home/cmccabe/Desktop/comparison/validation/files/*_*.txt files and know that there will be a corresponding file in /home/cmccabe/Desktop/comparison/validation/files/ with a name that has the same unique string before the first underscore character and ending in the string .vcf? Note that the name from post #3 quoted above marked in red does not end in .vcf. Was that a typo, or do some names in that directory not end in .vcf?

Is the assumption that there is only one file with the string before the first underscore in both of those directories correct? Or, do you want a script that depends on you creating a file named /home/cmccabe/Desktop/comparison/ref_val/out that contains lines containing three values (file1 filename, file2 filename, and output file filename)?

Is it OK to use an awk script that processes all 250 sets of input files in one invocation instead of invoking awk 250 times?

All of the REF files end in .txt and are located in a folder at /home/cmccabe/Desktop/comparison/reference/10bp.

All of the VAL files end in .vcf and are located in a folder at /home/cmccabe/Desktop/comparison/validation/files. I did have a typo in post #3.

Yes, there will only be one file in each separate directory with the string before the first _. So there is no need for an out file other then to know which samples were processed.

There may not always be 250 sets of input, that # is variable, but yes all of them can be processed at once rather than each set individually. is that what you mean? Thank you .

No, unless the statement on REF's contents in post#1 was not true. Given it IS true, FN assumes three file names ending in .txt

They are not. The proposal assumes that for every .txt- file name's prefix ID a respective .vcf file exists, possibly in another path (as mentioned in my post).

The operation of the awk script is not the topic of this thread, nor is the desired output.

Please! check if any of the proposals hitherto provide the needed input file pairs, might be adaptable to also provide the needed output file name, and comment on their aptitude.

And, please please please, get some structure into your future requests and relieve us from guessing!