Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Match and store numerical prefix to update files
Top Forums Shell Programming and Scripting Match and store numerical prefix to update files Post 302990884 by cmccabe on Thursday 2nd of February 2017 08:17:24 AM
Old 02-02-2017
Match and store numerical prefix to update files
In the bash below the unique headers of each vcf.gz are stored in a text file with the same name. That is if 16-0000-file.vcf.gz was used the header text file would be 16-0000-file_header.txt.
There can be multiple vcf.gz in a directory, usually 3, that I need to fix the header in each file before further processing it. My question is how can I match each text file with its vcf.gz and pass the stored variables of each to the reheader code ?

In the below I strip off the unique numerical prefix 16-0000 from both the vcf.gz and text file, but am not sure how to match the two files

Code:
IAm=${0##*/}

InDir1='/home/cmccabe/Desktop/NGS/test'
InDir2='/home/cmccabe/Desktop/NGS/test'
OutDir='/home/cmccabe/Desktop/NGS/test'

cd "$InDir1"
for file1 in *.txt
do	# Grab file prefix.
	p=${file1%%_*}

	# Find matching file2.
	file2=$(printf '%s' "$InDir2/$p"_*.vcf.gz)
	if [ ! -f "$file2" ]
	then	printf '%s: No single file matching %s found.\n' "$IAm" \
		    "$file1" >&2
		continue
	fi

# store matches
    out=${file1##*/} && ${file2##*/}

vcf.gz in directory (file2)
Code:
16-0000-file1.vcf.gz
16-0001-file2.vcf.gz
16-0002-file3.vcf.gz

matching text file in directory (file1)
Code:
16-0000-file1_header.txt
16-0001-file2_header.txt
16-0002-file3_header.txt

So the contents of 16-0000.txt would be used to update 16-0000.vcf.gz using the code below.

reheader code
Code:
# edit the header
logfile=/home/cmccabe/Desktop/NGS/test/process.log
for f in /home/cmccabe/Desktop/NGS/test/*.vcf.gz ; do
     echo "Start vcf add header creation: $(date) - file: $f"
     bname=`basename $f`
     pref=${bname%%.vcf.gz}
      bcftools reheader -h $file1 $file2 > ${pref}_fixed.vcf.gz
     echo "End add header creation: $(date) - file: $f"
done >> "$logfile"

Last edited by cmccabe; 02-02-2017 at 01:40 PM.. Reason: fixed format, added details to bash
 

10 More Discussions You Might Find Interesting

1. Programming

Fuzzy Match Logic for Numerical Values

I have searched the internet (including these forums) and perhaps I'm not using the right wording. What I'm looking for is a function (preferably C) that analyzes the similitude of two numerical or near-numerical values, and returns either a true/false (match/nomatch) or a return code that... (4 Replies)
Discussion started by: marcus121
4 Replies

2. Shell Programming and Scripting

Match Pattern and store next value into array

Hi, I am trying to write a script which parses a log file and will eventually put the values in an array so that I can perform some math on it. In this file I am only interested in the last 200 lines so here is the command I use to display the contents in a manageable manner. tail -200... (3 Replies)
Discussion started by: Filter500
3 Replies

3. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

4. Shell Programming and Scripting

Extract Uniq prefix from a start and end prefix

Dear All, assume i have a file with content: <Start>6000</Start> <Stop>7599</Stop> the output is: 6000 7000 7100 7200 7300 7400 7599 how should we use any awk, sed, perl can do this task, means to extract the uniq prefixes from the start and stop prefix. Thanks Jimmy (3 Replies)
Discussion started by: jimmy_y
3 Replies

5. Shell Programming and Scripting

Bash to match and store line as variable

The bash below loops through a specific directory dir and finds and writes the oldest folder to a variable called $filename. #!/bin/bash # oldest folder stored as variable for analysis, version log created, and quality indicators matched to run dir=/home/cmccabe/Desktop/NGS/test find... (2 Replies)
Discussion started by: cmccabe
2 Replies

6. Shell Programming and Scripting

awk to update file based on partial match in field1 and exact match in field2

I am trying to create a cronjob that will run on startup that will look at a list.txt file to see if there is a later version of a database using database.txt as the source. The matching lines are written to output. $1 in database.txt will be in list.txt as a partial match. $2 of database.txt... (2 Replies)
Discussion started by: cmccabe
2 Replies

7. Shell Programming and Scripting

Bash to add portion of text to files in directory using numerical match

In the below bash I am trying to rename eachof the 3 text files in /home/cmccabe/Desktop/percent by matching the numerical portion of each file to lines 3,4, or 5 in /home/cmccabe/Desktop/analysis.txt. There will always be a match between the files. When a match is found each text file in... (2 Replies)
Discussion started by: cmccabe
2 Replies

8. Shell Programming and Scripting

awk to update file with numerical difference if condition is met

In the file1 below if $9 and $12 are . (dot) then the value in $8 of file1 is used as a key (exact match) to lookup in each $2 of file2, when a match is found then the value of $4 in file1 is used to look for a range match within +/- 50 using the values in $4 and after in file2. The number of... (9 Replies)
Discussion started by: cmccabe
9 Replies

9. Shell Programming and Scripting

awk move select fields to match file prefix in two directories

In the awk below I am trying to use the file1 as a match to file2. In file2 the contents of $5,&6,and $7 (always tab-delimited) and are copied to the output under the header Quality metrics. The below executes but the output is empty. I have added comments to help and show my thinking. Thank you... (0 Replies)
Discussion started by: cmccabe
0 Replies

10. Shell Programming and Scripting

Bash to update file on prefix match in two directories

I am trying to use bash to loop through a directory /path/to/data using a prefix match from /path/to/file. That match is obtained and works using the code below (in green)... what I can not seem to do is populate or update the corresponding prefix_file.txt in /path/to/data with the values in each... (3 Replies)
Discussion started by: cmccabe
3 Replies
All times are GMT -4. The time now is 08:59 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy