awk to match and apply condtions to matchijng files in directories
I am trying to merge the below awk, which compares two files looking for a match in $2 and then prints the line if two conditions are meet.
awk
This code was improved and provided by @RavinderSingh13, thank you very much. I have ~500 files to process so I wanted to use all .txt files in /home/cmccabe/Desktop/comparison/missing and compare them to each matching numerical prefix in /home/cmccabe/Desktop/comparison/test_tvc all ending in .bed. Each filename in a directory will have a common numerical prefix:
So if there are three files, the three .txt files in home/cmccabe/Desktop/comparison/missing will look like:
and the three .bed files in /home/cmccabe/Desktop/comparison/test_tvc will look like:
So F113.txt would be compared to F113_tvc.bed, the matching numerical prefix is F113.
If a match between the $2 values in eaach file is made and both conditions if($10>30 && $11>49 are meet, then the matching line from the .txt file is printed in the out under Match in both files and meet criteria. If no match is found or the criterias is not meet then the line in the .txt is printed in the out under Missing in comparison:.
The below code provided by @Don Cragun works great but since my data has changed a bit I made some updates to it:
updated version which does not run with comments marked by --
I am not sure how to perform the two if statements on the matching $2 values. Below are two sample input files as well as the desired output.
file1 (F113.txt)
file2 (F113.bed)
desired output
I hope I have included enough information and thank you .
Could you please try following and let me know if this helps.
Also I am not sure why you are taking awkcommand's output into a file? If this is the case then you shoulduse following command then.
Again, I haven't tested it all so there may be a chance to tweak it a bit, kindly check it and let me know how it goes then.
Thanks,
R. Singh
Last edited by RavinderSingh13; 10-16-2016 at 05:34 AM..
Reason: thanks to greet_sed for letting me know the missing $ in file1 variable.
This User Gave Thanks to RavinderSingh13 For This Post:
The output of the awk is stored for each par of files that are compared as that match/difference is important to know. The command does run but there is no output file created for each. So if there are 3 file compared say:
From /home/cmccabe/Desktop/comparison/missing the file F113.txt is compared to /home/cmccabe/Desktop/comparison/test_tvc file F113_tvc.bed the matches and differences are the stored in the output at /home/cmccabe/Desktop/comparison/final called F113_final.txt
From /home/cmccabe/Desktop/comparison/missing the file H123.txt is compared to /home/cmccabe/Desktop/comparison/test_tvc file H123_tvc.bed the matches and differences are the stored in the output at /home/cmccabe/Desktop/comparison/final called H123_final.txt
From /home/cmccabe/Desktop/comparison/missing the file S111.txt is compared to /home/cmccabe/Desktop/comparison/test_tvc file S111_tvc.bed the matches and differences are the stored in the output at /home/cmccabe/Desktop/comparison/final called S111_final.txt
Could you please try following and let me know if this helps you.
Also files out_no_match_found_valuesandMatch_in_both_files_and_meet_criteria in path /home/cmccabe/Desktop/comparison/missing couldn't be seen, because there is no complete path given for those files so in case you need them
in any other path please use absolute path eg-->/path/to/file/out_no_match_found_values for these output files and then it should fly. Let me know how it goes then.
Thanks,
R. Singh
Last edited by RavinderSingh13; 10-16-2016 at 05:34 AM..
Reason: thanks to greet_sed for letting me know the missing $ in file1 variable.
This User Gave Thanks to RavinderSingh13 For This Post:
In the awk below I am trying to use the file1 as a match to file2. In file2 the contents of $5,&6,and $7 (always tab-delimited) and are copied to the output under the header Quality metrics. The below executes but the output is empty. I have added comments to help and show my thinking. Thank you... (0 Replies)
I am trying to look for $2 of file1 (skipping the header) in $2 of file2 (skipping the header) and if they match and the value in $10 is > 30 and $11 is > 49, then print the line from file1 to a output file. If no match is foung the line is not printed. Both the input and output are tab-delimited.... (3 Replies)
Greetings Experts,
I am on AIX and in process of creating a re-startable script that connects to Oracle and executes the statements. The sample contents of the file1 is
CREATE OR REPLACE VIEW DB_V.TAB1 AS SELECT * FROM DB_T.TAB1;
....
CREATE OR REPLACE VIEW DB_V.TAB10 AS SELECT * FROM... (9 Replies)
Hi,
My script works fine when I have both input files in the same directory but when I put on of the input file in another directory, the output does not show up.
SCRIPT:
awk '
BEGIN {
OFS="\t"
out = "File3.txt"}
NR==FNR && NF {a=$0; next}
function print_77_99() {
if... (3 Replies)
Write a Unix shell script named 'mode' that accepts two or more arguments, a file mode, a command and an optional list of parameters and performs the given command with the optional parameters on all files with that given mode.
For example, mode 644 ls -l should perform the command ls -l on all... (5 Replies)
HI there. My teacher asked us to write a code for this question
Write a Unix shell script named 'mode' that accepts two or more arguments, a file mode, a command and an optional list of parameters and performs the given command with the optional parameters on all files with that given mode. ... (1 Reply)
Hi,
I have a bunch of records within a directory where each one has this form:
(example file1)
1 2 50 90 80 90 43512 98 0909 79869 -9 7878 33222 8787 9090 89898 7878 8989 7878 6767 89 89 78676 9898 000 7878 5656 5454 5454
and i want for all of these files to be... (3 Replies)
Hi Guys,
Can you tell me if unix permissions apply to sub dirs?
Dir is /home/ops/batch/files/all
/home is rwxrwxrwx
ops is rwxrwxrwx
batch is rwxr-wr-w
files is rwxrwxrwx
all is rwxrwxrwx
Having problems writing to all (does the userid nee to be the batch owner... (1 Reply)
Hi All,
I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs.
How would I do this in a script?
I have the following script called "csvtabs":
awk 'BEGIN {
FS... (4 Replies)
Hey,
I'm very new to AWK and am trying to write a script that counts the number of files in all subdirectories. So, basically, my root has many subdirectories, and each subdirectory has many files.
How can I get the total count? I haven't been able to figure out how to loop through the... (1 Reply)