Bash to extract file prefix and from input to use in output


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash to extract file prefix and from input to use in output
# 1  
Old 10-25-2017
Bash to extract file prefix and from input to use in output

In the bash below which does execute I am trying to extract the contents of ${id} is 1234, as ${id} stores the variable that changes each time.
After the path is removed the contents of ${id} are stored in pref, so they can be used in the output. Currently I am not able to extract the 1234 in the for and can not seem to figure it out. Thank you Smilie.

Code:
awk: cmd. line:2: fatal: cannot open file `C:/path/to/file/.txt
.hg19_multianno.txt' for reading (No such file or directory)

Code:
for f in C:/path/to/file/${id}.txt.hg19_multianno.txt ; do
     bname=`basename $f`
     pref=${bname%%$.*.txt}
     awk -v F=49 -v T=96 '
BEGIN{FS=OFS="\t"}
{ b=T+1
t=T<NF?T:NF
for(i=F;i<NF-t+F;i++) $i=$(b++)
NF=--i}1' $f > C:/path/to/file/${pref}_dbremoved.txt
done

desired C:/path/to/file/${pref}_dbremoved.txt
Code:
1234_dbremoved

# 2  
Old 10-25-2017
Quote:
Originally Posted by cmccabe
[...]
Code:
for f in C:/path/to/file/${id}.txt.hg19_multianno.txt ; do
     bname=`basename $f`
     pref=${bname%%$.*.txt}
     awk -v F=49 -v T=96 '
BEGIN{FS=OFS="\t"}
{ b=T+1
t=T<NF?T:NF
for(i=F;i<NF-t+F;i++) $i=$(b++)
NF=--i}1' $f > C:/path/to/file/${pref}_dbremoved.txt
done

Let's reflect in what's highlighted. The id variable needs to be populated before it can be used. Now, let's assume it contains a value that ultimately will result in a file path. A loop is not necessary to iterate over what it would just be one item.
This User Gave Thanks to Aia For This Post:
# 3  
Old 10-25-2017
There may be multiple ${id} values in the directory, however each unique ${id} is stored in a file called list, one per line, in the same path. So if 1234, 5678, 9123 are each ${id} the list will look like:

list.txt
Code:
1234.txt
5678.txt
9123.txt

Would this file need to be used to populate each id? I am not sure if that helps. Thank you Smilie.
# 4  
Old 10-25-2017
Start by reading the file named list that contains the information.

Code:
dir_path=$PWD
while IFS= read line
do
    id=${line%.*}
    echo "$dir_path/${id}.whatever_else"
done < list

And build upon that.
This User Gave Thanks to Aia For This Post:
# 5  
Old 10-26-2017
So the portion in bold identifies ${id} as 1234 perfectly, however it does not store it so the for loop can use it and output the file with the 1234. Do I not need a loop even though ${id} could be multiple? Thank you Smilie.

Code:
# read id from target.txt
dir_path=C:/Users/cmccabe/Desktop/annovar
while IFS= read line
do
    id=${line%.*}
    echo "$dir_path/${id}.hg19_multianno.txt"
done < target.txt

# remove -dbnsfp33a fields 49-96 from multianno
${id}=(line)
for f in 'C:\Users\cmccabe\Desktop\annovar\${id}.txt.hg19_multianno.txt' ; do
     bname=`basename $f`
     pref=${bname%%$.*.txt}
     awk -v F=49 -v T=96 '
BEGIN{FS=OFS="\t"}
{ b=T+1
t=T<NF?T:NF
for(i=F;i<NF-t+F;i++) $i=$(b++)
NF=--i}1' $f > C:/Users/cmccabe/Desktop/annovar/${pref}_dbremoved.txt
done


Last edited by cmccabe; 10-26-2017 at 10:04 AM.. Reason: added to scipt
# 6  
Old 10-26-2017
Work from this point.
Code:
dir_path=C:/Users/cmccabe/Desktop/annovar
while IFS= read line
do
    id=${line%.*}
    awk -v F=49 -v T=96 '
        BEGIN{ FS=OFS="\t" }
        { b=T+1
           t=T<NF?T:NF
           for(i=F;i<NF-t+F;i++) {
              $i=$(b++)
              NF=--i}
         }1' "$dir_path/${id}.hg19_multianno.txt" > "$dir_path/${id}_dbremoved.txt"
done < target.txt

Modified on the fly. Not tested.
This User Gave Thanks to Aia For This Post:
# 7  
Old 10-31-2017
Thank you very much Smilie.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash to update file on prefix match in two directories

I am trying to use bash to loop through a directory /path/to/data using a prefix match from /path/to/file. That match is obtained and works using the code below (in green)... what I can not seem to do is populate or update the corresponding prefix_file.txt in /path/to/data with the values in each... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

Bash to add unique prefix to extracted zip folder

In the bash below in each .zip there is a folder to be extracted Variants that I am trying to make unique by adding the prefix, before the _ from the .zip. The script does execute, but the prefix is not added to the extracted folder. Rather the Variants folder is added to each file within it. Thank... (1 Reply)
Discussion started by: cmccabe
1 Replies

3. Shell Programming and Scripting

Extract Uniq prefix from a start and end prefix

Dear All, assume i have a file with content: <Start>6000</Start> <Stop>7599</Stop> the output is: 6000 7000 7100 7200 7300 7400 7599 how should we use any awk, sed, perl can do this task, means to extract the uniq prefixes from the start and stop prefix. Thanks Jimmy (3 Replies)
Discussion started by: jimmy_y
3 Replies

4. Shell Programming and Scripting

Extract the lines from input file

This is the sample input file b 05/Jul/2010:07:00:10 a 05/Jul/2010:06:00:10 b 05/Jul/2010:07:00:10 c 05/Jul/2010:07:10:10 d 05/Jul/2010:08:00:10 e 05/Jul/2010:09:00:10 f 05/Jul/2010:10:00:10 h 05/Jul/2010:11:00:10 i 05/Jul/2010:12:00:10 j ... (9 Replies)
Discussion started by: sandy1028
9 Replies

5. Shell Programming and Scripting

Dynamic output file generation using a input text file with predefined output format

Hi, I have two files , one file with data file with attributes that need to be sent to another file to generate a predefined format. Example: File.txt AP|{SSHA}VEEg42CNCghUnGhCVg== APVG3|{SSHA}XK|"password" AP3|{SSHA}XK|"This is test" .... etc --------- test.sh has... (1 Reply)
Discussion started by: hudson03051nh
1 Replies

6. Shell Programming and Scripting

Extract date from file header and prefix it to all lines

Hello All, I have a file in the following format. I want to extract the date(020090930, 020090929) in the string "STPAGE020090930" and "STPAGE020090929" and prefix it to all lines below them. The output must be put into a new file. STPAGE020090930 xyzz aalc... (3 Replies)
Discussion started by: john2022
3 Replies

7. Shell Programming and Scripting

Prefix the current date and time to the output of ps

Hi, I need to write a script, that will take the current date, time, and the output from # ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm and spit it to a file, so it'll look like this... PID TID CLS RTPRIO NI PRI PSR %CPU STAT WCHAN COMMAND 1 1 TS... (2 Replies)
Discussion started by: Bloke
2 Replies

8. Shell Programming and Scripting

Please help: Extract filtered output from following input

Following is input: <P align="justify" ><FONT size="+1" color="#221E1F">the tiny bundles of hairs that protrude from them. Waves in the fluid of the inner ear stimulate the hair cells. Like the rods and cones in the eye, the hair cells convert this physical stimulation into neural im<FONT... (1 Reply)
Discussion started by: parshant_bvcoe
1 Replies

9. UNIX for Dummies Questions & Answers

extract a part of a path like (input: /etc/exp/home/bin ====> output: exp)

Hi, I need to make some extraction . with the following input to get the right output. input: /etc/exp/home/bin ====> output: exp and input: aex1234 ===> output: ex Thanks for your help, (4 Replies)
Discussion started by: yeclota
4 Replies

10. Shell Programming and Scripting

how to extract paragraphs from file in BASH script followed by prefix ! , !! and !!!

I]hi all i am in confusion since last 2 days :( i posted thraed yesterday and some friends did help but still i couldnt get solution to my problem let it be very clear i have a long log file of alkatel switch and i have to seperate the minor major and critical alarms shown by ! , !! and !!!... (6 Replies)
Discussion started by: nabmufti
6 Replies
Login or Register to Ask a Question