Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

loop through lines and save into separate files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting loop through lines and save into separate files
# 1  
Old 11-01-2011
loop through lines and save into separate files
I have two files:
file-gene_families.txt that contains 30,000 rows of 30 columns. Column 1 is the ID column and contains the
Code:
Col1                  Col2  Col3 ...
One gene-encoded CBPs ABC  111 ...
One gene-encoded CBPs ABC  222 ...
One gene-encoded CBPs ABC  212 ...
Two gene encoded CBPs EFC   223 ...
Two gene encoded CBPs EFC   133 ...
Two gene encoded CBPs EFC   103 ...
Two gene encoded CBPs EFC   323 ...
Three gene(encoded) CBPs CGC  20 ...
Four gene/encoded (CBPs) GGH NULL ...
Four gene/encoded (CBPs) GGH 0 ...
Four gene/encoded (CBPs) GGH 1 ... 
Four gene/encoded (CBPs) GGH 2 ...
Four gene/encoded (CBPs) GGH 3 ...
Four gene/encoded (CBPs) GGH 56 ...

and
file-group.list.
Code:
One gene-encoded CBPs
Two gene encoded CBPs
Three gene(encoded) CBPs
Four gene/encoded (CBPs)

I want separate file-gene_families.txt based on the file-group.list using the each line of file-group.list as the file names of the output, substitute these brackets space and slash with hyphen "-".
Code:
One-gene-encoded-CBPs.tmp
Two-gene-encoded CBPs.tmp
Three-gene-encoded-CBPs.tmp
Four-gene-encoded-CBPs.tmp

for example in One-gene-encoded-CBPs.tmp
Code:
One gene-encoded CBPs ABC  111 ...
One gene-encoded CBPs ABC  222 ...
One gene-encoded CBPs ABC  212 ...

Could not get my script working. Can someone help me out?
Code:
#!/usr/bin/bash
IFS=$'\n'
for line in $(cat At-GeneFamily-Unique-group.list)
do
name=$(sed 's/\ |\//-/g' $line)
grep $line gene_families.txt >> $name.tmp
done

Thanks a lot!
YF
# 2  
Old 11-02-2011
Use ksh to run the script ..
Code:
while read i
do
        name=$(echo $i | sed 's,(, ,g;s,),,g;s,/, ,g;s, ,-,g')
        grep "$i" file-gene_families.txt > ${name}.tmp
done<file-group.list

This User Gave Thanks to jayan_jay For This Post:
yifangt
# 3  
Old 11-02-2011
Code:
nawk '{print $0 >> $1}' inputfile

# 4  
Old 11-02-2011
Thanks to you both! It works like a charm!
Raj, could you explain the script a little more detail? So simple and worked except some error, e.g. for those the first columns contains "/" did not go through.
Code:
$0 for the whole row,
$1 for the first column

What's the trick of the ">>" ? I am trying to combine the "sed" part to get rid of those special characters for the output file name.
Thanks a lot again!
YF
# 5  
Old 11-02-2011
Are columns TAB separated?
Based on your sample file, you have embedded space in what you call a 'first column' in the file-gene_families.txt
# 6  
Old 11-02-2011
Yes, they are TAB separated.
That's why I want the embedded space replaced with hyphen in the output files. Not fully understand Raj's script, Jayan's works good though.
Thanks. YF
# 7  
Old 11-02-2011
not tested...
Code:
nawk 'FNR==NR{f2[$0];next} $1 in f2 {a=$1;gsub("[ /()] ","-",a);out=(a ".txt"); print >> out;close(out)}' FS='\t' file-group.list file-gene_families.txt

Last edited by vgersh99; 11-02-2011 at 11:10 AM.. Reason: forgot to add FS='\t'
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Awk: output lines with common field to separate files

Hi, A beginner one. my input.tab (tab-separated): h1 h2 h3 h4 h5 item1 grpA 2 3 customer1 item2 grpB 4 6 customer1 item3 grpA 5 9 customer1 item4 grpA 0 0 customer2 item5 grpA 9 1 customer2 objective: output a file for each customer ($5) with the item number ($1) only if $2 matches... (2 Replies)
Discussion started by: beca123456
2 Replies

2. Shell Programming and Scripting

REGEX to separate paths by whitespace and do a loop

I am trying to do in a single line to take a list of paths separated by whitespace and then loop thru all the paths that were wrote but my regex is not working, I have echo {3} | sed 's/ //g' | while read EACHFILE do ..... But for some reason is only taking always the first path that I... (7 Replies)
Discussion started by: jorgejac
7 Replies

3. Shell Programming and Scripting

how to write on separate lines?

Hello friends, I have a file "a.txt" its contents ----------------- pid 4075 (caiopr) shmat(1929379932, 0x0000000000000000, 0) = 0x00000000ff030000 (errno 0) pid 4075 (caiopr) shmdt(0x00000000ff030000) = 144 (errno 0) pid 4075 (caiopr) shmctl(1929379932, IPC_RMID) pid 4205 (cau9cli.exe)... (2 Replies)
Discussion started by: Sunusernewbie
2 Replies

4. Shell Programming and Scripting

Combine the lines from separate text files

Hi All, I have three separate text files which has only one line and i want to combine these lines in one text file which will have three lines. cat file1.txt abc cat file2.txt 1265 6589 1367 cat file3.txt 0.98 0.36 0.5 So, I want to see these three lines in the... (9 Replies)
Discussion started by: senayasma
9 Replies

5. Shell Programming and Scripting

Sed save changes to same file in loop

I have got problems saving sed changes to the same file in a loop. Basically I want the delimited value in every line of the file to be set to blank according to the value stored in var. var can be changed anytime. I do not have sed -i and i've tried to mv the file. Any other ideas? My file... (8 Replies)
Discussion started by: alienated
8 Replies

6. Shell Programming and Scripting

extract nth line of all files and print in output file on separate lines.

Hello UNIX experts, I have 124 text files in a directory. I want to extract the 45678th line of all the files sequentialy by file names. The extracted lines should be printed in the output file on seperate lines. e.g. The input Files are one.txt, two.txt, three.txt, four.txt The cat of four... (1 Reply)
Discussion started by: yogeshkumkar
1 Replies

7. Shell Programming and Scripting

Concatenating lines of separate files using awk or sed

For example: File 1: abc def ghi jkl mno pqr File 2: stu vwx yza bcd efg hij klm nop qrs I want the reult to be: abc def ghistu vwx yza jkl mno pqrbcd efg hij klm nop qrs (4 Replies)
Discussion started by: tamahomekarasu
4 Replies

8. Shell Programming and Scripting

Direct the invalid lines to a separate files

Hi, I have a pipe delimited file with 1 million records. I need to validate each line by counting the number of delimiters, if any line fails to have the specified number of delimiters, taat line has to be sent to a reject file. Kindly suggest. if code provided, it is highly appreciated, and... (22 Replies)
Discussion started by: anandapani
22 Replies

9. Shell Programming and Scripting

counting the lines of diff files in loop

i have two file. i want to count the lines of each file one by one in loop and compare it. can any one pls help me on this? (1 Reply)
Discussion started by: Aditya.Gurgaon
1 Replies

10. Shell Programming and Scripting

diff lines from 2 files in a while loop

Okie I have two files. file1 with input asdf_s45 fdsa_s20 jkl_s32 lkj_s3 and file2 with input asdf_s44 fdsa_s19 jkl_s31 lkj_s2 now I have counted the total number of lines in the file and put it in a variable so num_lines=4 now I have a while loop to repeat a diff command... (6 Replies)
Discussion started by: bigboizvince
6 Replies
Login or Register to Ask a Question