Bash to update files to be used by awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash to update files to be used by awk
# 1  
Old 10-03-2016
Bash to update files to be used by awk

In the below bash there are 6 .txt files in /home/cmccabe/Desktop/comparison/ref_val/ that are being updated from the 6 .vcf files in /home/cmccabe/Desktop/comparison/validation/files/*.vcf. The awk in the post updates the files with the information, however the files are not being updated so the awk stalls. Each filename will have the portion before the _ that matches between /home/cmccabe/Desktop/comparison/ref_val/*.txt and /home/cmccabe/Desktop/comparison/validation/files/*.vcf

Example of file in /home/cmccabe/Desktop/comparison/ref_val/*.txt
Code:
F13_epilepsy_filtered_comparison.txt

this file is updated from /home/cmccabe/Desktop/comparison/validation/files/*.vcf
Code:
F13_epilepsy_filtered.vcf


FN= /home/cmccabe/Desktop/comparison/ref_val/*.txt

matching files used to update FN
/home/cmccabe/Desktop/comparison/validation/files/*.vcf


example of vcf file from (this is the file that update FN)
/home/cmccabe/Desktop/comparison/validation/files/*.vcf
Code:
chr1    43395635    43395635    C    T    exonic    SLC2A1    .    synonymous SNV    SLC2A1:NM_006516:exon5:c.588G>A:p.P196P    rs2229682    0.23    0.12    0.024    0.21    0.08    0.19    0.15    0.18    0.044    0.19    0.074    0.23    0.21    0.19    0.19    0.15    0.049    0.2    0.12    -0.1558    -0.594    .    .    .    .    .    .    .    .    .    .    .    .    Benign    not_specified    RCV000081436.5    MedGen    CN169374    GOOD    399    het    19

example of text file from (this is the file being updated)
/home/cmccabe/Desktop/comparison/ref_val/*.txt

Bash
Code:
#!/bin/bash
     for FN in /home/cmccabe/Desktop/comparison/ref_val/*.txt; do
awk '
BEGIN {FS = OFS = "\t"
}
NR == 1 {
outfile = FILENAME
}
FNR == NR {
o[i[++ic] = $1 OFS $2 OFS $3] = $0
}
{if($2 OFS $4 OFS $5 in o)
o[$2 OFS $4 OFS $5] = $1 OFS $2 OFS $4 OFS $5 OFS $6 OFS $7 OFS $50 OFS $51 OFS $52 OFS $53
}
END {for(j = 1; j <= ic; j++)
print o[i[j]] > outfile
}' $FN ${FN%%_*}/home/cmccabe/Desktop/comparison/validation/files/*.vcf
    done

desired output (this is FN before updating)
Code:
Match:
9943666    C    T
161281245    T    C
138662273    G    A
64142859    C    T

this is FN after updating
Code:
Match:
chr16    9943666    C    T    exonic    GRIN2A    GOOD    343    het    18
chr5    161281245    T    C    exonic    GABRA1    GOOD    197    het    13
chr9    138662273    G    A    exonic    KCNT1    GOOD    275    het    15
chr3    64142859    C    T    exonic    PRICKLE2    GOOD    355    het    23

If no match is found in /home/cmccabe/Desktop/comparison/validation/files/*.vcfto update FN, then it it skipped and the next line is processed. I hope I have included everything and thank you Smilie.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash FPATH code update

In this post at 302451613-post2.html the link to the code comes up not found. The thread is closed, so I was unable to ask on the thread itself and I do not have enough posts yet to send a private message (or write out a proper html link). Does the author (jim mcanamara) have an updated link? ... (2 Replies)
Discussion started by: matthewpersico
2 Replies

2. Shell Programming and Scripting

Bash-awk to process thousands of files

Hi to all, I have thousand of files in a folder with names with format "FILE-YYYY-MM-DD-HHMM" for what I want to send the following AWK command awk '/Code.*/' FILE-2014* I'd like to separate all files that have the same date to a folder named with the corresponding date. For example, if I... (7 Replies)
Discussion started by: Ophiuchus
7 Replies

3. Red Hat

RedHat 5 update 9 BASH update issue

Hi i want to update the BASH because of the "shell shock" vulnerability. my RedHat 5 is clean install with the default mirror site. when im running the command: yum update bash im getting a message saying there is no update. you can see in the attach picture... what am i doing wrong? is... (4 Replies)
Discussion started by: guy3145
4 Replies

4. Solaris

Solaris bash vulnerability update

The patch blog has: https://blogs.oracle.com/patch/entry/solaris_idrs_available_on_mos information on dealing with bash 'shellshock' vulnerability. (3 Replies)
Discussion started by: jim mcnamara
3 Replies

5. Shell Programming and Scripting

Passing multiple files to awk for processing in bash script

Hi, I'm using awk command in bash script. I'm able to pass multiple files to awk for processing.The code i can use is as below(sample code) #!/bin/bash awk -F "," 'BEGIN { ... ... ... }' file1 file2 file3 In the above code i'm passing the file names manually and it is fine till my... (7 Replies)
Discussion started by: shree11
7 Replies

6. Shell Programming and Scripting

Update a mysql column via bash script

Hello, I want to check the value of all MySQL columns.(column name is "status") via bash script. If value is "0" at I want to make only single column value to "1" I have many "0" values on mysql database(on "status" column) "0" means it is a draft post. I want to publish a post. I... (2 Replies)
Discussion started by: tara123
2 Replies

7. Shell Programming and Scripting

Update ksh .profile to launch bash

Hi I don't have chsh option. I want to launch bash instead of ksh ( or launch bash from ksh .profile) how can I do this ? (1 Reply)
Discussion started by: Sivaswami
1 Replies

8. UNIX for Advanced & Expert Users

Bash executing Orcale Update statement

Hi All, Using Solaris box bash-3.00$ echo $BASH_VERSION 3.00.16(1)-release I have a real bummer of a bug, basically Im running a bash script that executes a bash function "dbase_sql". The bash function accepts a parameter in the form of an Oracle update statement eg ... (1 Reply)
Discussion started by: satnamx
1 Replies

9. Shell Programming and Scripting

Constant update echo in BASH

Hi all, Basically Im trying to put the current time in a script in BASH. Tried the watch command, but its not really what I want. I will have lots of things in this script, current date and time being just a few). Any ideas? (4 Replies)
Discussion started by: mikejreading
4 Replies

10. Shell Programming and Scripting

Bash and Awk for creating directories and moving files

I have a security system that FTPs the camera files to my machine, however I want to sort the pictures (taken every 30s) into directories by hour. Every picture uses the following file format. yymmddhhmmsstt.jpg (where tt is the milliseconds) I am thinking the for loop is best for file... (11 Replies)
Discussion started by: Kiint
11 Replies
Login or Register to Ask a Question