Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk separate files to one directory

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk separate files to one directory
# 1  
Old 10-12-2015
awk separate files to one directory
I am trying to output all files that are made by this awk to a specific directory.

Code:
 awk -F '[ :]' '{f = $3 ".txt"; print > f}' input.txt

Since the actual data has several hundred files I redirect the output (well tried to) to a directory.

Code:
awk -F '[ :]' '{f = $3 ".txt";  close($3 ".txt")} print > f}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed > /home/cmccabe/Desktop/panels/BMF
 bash: /home/cmccabe/Desktop/panels/BMF: Is a directory

So I tried (thought defining the variable may help) the below but it didn't help. Thank you Smilie.

Code:
awk -F '[ :]' '{f = $3 ".txt";  close($3 ".txt")} print > f}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed > /home/cmccabe/Desktop/panels/BMF/"$f".txt; 
bash: /home/cmccabe/Desktop/panels/BMF: Is a directory

# 2  
Old 10-12-2015
Hello cmccabe,

Following may help you in same.
Code:
awk -F '[ :]' '{f = $3 ".txt";  close($3 ".txt")} print > "/home/cmccabe/Desktop/panels/BMF/" f}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
cmccabe
# 3  
Old 10-12-2015
Thank you for the code Smilie, here is the error I get.


Code:
awk -F '[ :]' '{f = $3 ".txt";  close($3 ".txt")} print > "/home/cmccabe/Desktop/panels/BMF/" f}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed
awk: line 1: syntax error at or near print
awk: line 1: extra '}'

If I remove the close($3 ".txt")} it works great. Thank you Smilie.
Last edited by cmccabe; 10-12-2015 at 11:31 AM.. Reason: added edit
# 4  
Old 10-12-2015
Hello cmccabe,

Yes, you could try below.
Code:
 awk -F '[ :]' '{f = $3 ".txt";  close($3 ".txt");print > "/home/cmccabe/Desktop/panels/BMF/" f}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed

But actually speaking you should below for same.
Code:
 awk -F '[ :]' '{f = $3 ".txt";print > "/home/cmccabe/Desktop/panels/BMF/" f;  close(f);}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed

Hope this helps.

Thanks,
R. Singh
# 5  
Old 10-12-2015
Not really... There are a couple of problems here.
The standards don't define the precedence between output redirection and string concatenation in awk print commands. So, although it appears to work on your system, it isn't portable. And, if you write to one string, as in:
Code:
print > ("/home/cmccabe/Desktop/panels/BMF/" f)

you need to close that same string; so:
Code:
close(f)

is a no-op because f has never been opened. So, what you really need is something more like:
Code:
awk -F '[ :]' '
{f = "/home/cmccabe/Desktop/panels/BMF/" $3 ".txt"
 print > f
 close(f)
}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed

Or, more efficiently:
Code:
cd /home/cmccabe/Desktop/panels/BMF
awk -F '[ :]' '
{f = $3 ".txt"
 print > f
 close(f)
}' BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed

# 6  
Old 10-12-2015
He/she'd need to add a path to the input file, then...
Last edited by RudiC; 10-12-2015 at 03:59 PM..
This User Gave Thanks to RudiC For This Post:
Don Cragun
# 7  
Old 10-12-2015
Thanks RudiC. Yes, it should have been:
Code:
srcdir="$PWD"
cd /home/cmccabe/Desktop/panels/BMF
awk -F '[ :]' '
{f = $3 ".txt"
 print > f
 close(f)
}' "$srcdir"/BMF_unix_loop_genes_IonXpress_008_150902_loop_genes_average_IonXpress_008_150902.bed

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Awk: output lines with common field to separate files

Hi, A beginner one. my input.tab (tab-separated): h1 h2 h3 h4 h5 item1 grpA 2 3 customer1 item2 grpB 4 6 customer1 item3 grpA 5 9 customer1 item4 grpA 0 0 customer2 item5 grpA 9 1 customer2 objective: output a file for each customer ($5) with the item number ($1) only if $2 matches... (2 Replies)
Discussion started by: beca123456
2 Replies

2. Shell Programming and Scripting

awk to create separate files but not include specific field in output

I am trying to use awk to create (in this example) 3 seperate text file from the unique id in $1 in file, if it starts with the pattern aa. The contents of each row is used to populate each text file except for $1 which is not needed. It seems I am close but not quite get there. Thank you :). ... (3 Replies)
Discussion started by: cmccabe
3 Replies

3. Programming

GFORTRAN sending .o files in separate directory

I am using gfortran and want to send .o files in separate directory. Currently I have the following in a bash script. The .mod files are handled ok, but the .o files are still created in the same directory as the source code fsrc="newunit.f08 math.f08 tString.f08 prout.f08 prary.f08... (1 Reply)
Discussion started by: kristinu
1 Replies

4. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

5. UNIX for Dummies Questions & Answers

awk to match multiple regex and create separate output files

Howdy Folks, I have a list that looks like this: (file2.txt) AAA BBB CCC DDD and there are 24 of these short words. I am matching these patterns to another file with 755795 lines (file1.txt). I have this code for matching: awk -v f2=file2.txt ' BEGIN { while(... (2 Replies)
Discussion started by: heecha
2 Replies

6. Shell Programming and Scripting

Apply 'awk' to all files in a directory or individual files from a command line

Hi All, I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs. How would I do this in a script? I have the following script called "csvtabs": awk 'BEGIN { FS... (4 Replies)
Discussion started by: ScKaSx
4 Replies

7. Shell Programming and Scripting

handling multiple files using awk command and wants to get separate out file for each

hai all I am new to the world of shell scripting I wanted to extract two columns from multiple files say around 25 files and i wanted to get the separate outfile for each input file tired using the following command to extract two columns from 25 files awk... (2 Replies)
Discussion started by: hema dhevi
2 Replies

8. Shell Programming and Scripting

Concatenating lines of separate files using awk or sed

For example: File 1: abc def ghi jkl mno pqr File 2: stu vwx yza bcd efg hij klm nop qrs I want the reult to be: abc def ghistu vwx yza jkl mno pqrbcd efg hij klm nop qrs (4 Replies)
Discussion started by: tamahomekarasu
4 Replies

9. Shell Programming and Scripting

Using AWK to separate data from a large XML file into multiple files

I have a 500 MB XML file from a FileMaker database export, it's formatted horribly (no line breaks at all). The node structure is basically <FMPXMLRESULT> <METADATA> <FIELD att="............." id="..."/> </METADATA> <RESULTSET FOUND="1763457"> <ROW att="....." etc="...."> ... (16 Replies)
Discussion started by: JRy
16 Replies

10. UNIX for Dummies Questions & Answers

Using Awk within awk to read all files in directory

I am wondering if anyone has any idea how to use an awk within awk to read files and find a match which adds to count. Say I am searching how many times the word crap appears in each files within a directory. How would i do that from the command prompt ... thanks (6 Replies)
Discussion started by: flevongo
6 Replies
Login or Register to Ask a Question