Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Writing a for loop to manipulate multiple files

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Writing a for loop to manipulate multiple files
# 1  
Old 09-06-2012
Writing a for loop to manipulate multiple files
Hi,
I have 1000 text files in a folder that are labeled data1.txt all the way to data1000.txt. I want to write a small script that manipulates the text files in this way:

(1) cut the 2nd and 9th columns of the text files
(2) sort by the numerical value in the 9th column
(3) then save the rows where the value in the 9th column is greater than 0.001 in a file that is called newdata*txt. (newdata1.txt to newdata1000.txt)

I was wondering if you could help me write a for loop script to achieve that. Thank you!
# 2  
Old 09-06-2012
Show some input and matching output please.
# 3  
Old 09-07-2012
Input:
Code:
1         rs3131972     742584    T        ADD     3016      1.031       0.4088       0.0006827
   1         rs2073813     743404    T        ADD     3004      1.048       0.5761       0.5646
   1        rs11564776     750163    C        ADD     3002      1.123       0.9788       0.0003277
   1        rs12562034     758311    A        ADD     3014      1.018       0.2118       0.8322
   1        rs12124819     766409    G        ADD     3005      1.058        0.918       0.3586
   1         rs2980300     775852    T        ADD     3016      1.045       0.5531       0.5802
   1        rs11240777     788822    A        ADD     3015      1.075        1.072       0.2836

Output:
Code:
rs11240777 0.2836
rs12124819 0.3586
rs2073813 0.5646
rs2980300 0.5802
rs12562034 0.8322

# 4  
Old 09-07-2012
Code:
for i in *.txt 
do
awk '$9 > 0.001{print $2, $9}' $i | sort -nrk2 > newdata$i
done

This User Gave Thanks to pamu For This Post:
evelibertine
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to do I manipulate a variable in a do loop?

This is my code cat /path/file | while read host do echo "\nPINGING $host" ip=`eval "$host | cut -c14-28"` echo $ip ping $ip sleep 2 done the file contains a list of hostnames and IP addresses in format HOSTNAMEXXXX,168.192.100.150 (10 Replies)
Discussion started by: ditnl555v
10 Replies

2. UNIX for Dummies Questions & Answers

[Solved] Writing a loop to changing the names of files in a directory

Hi, I would like to write a loop to change the names of files in a directory. The files are called data1.txt through data1000.txt. I'd like to change their names to a1.txt through a1000.txt. How do I go about doing that? Thanks! (2 Replies)
Discussion started by: evelibertine
2 Replies

3. UNIX for Dummies Questions & Answers

Writing a script to print the number of lines in multiple files

Hi I have 1000 files labelled data1.txt through data1000.txt. I want to write a script that prints out the number of lines in each txt file and outputs it in the following format: Column 1: number of data file (1 through 1000) Column 2: number of lines in the text file Thanks! (2 Replies)
Discussion started by: evelibertine
2 Replies

4. UNIX for Dummies Questions & Answers

Writing a loop to process multiple input files by a shell script

I have multiple input files that I want to manipulate using a shell script. The files are called 250.1 through 250.1000 but I only want the script to manipulate 250.300 through 250.1000. Before I was using the following script to manipulate the text files: for i in 250.*; do || awk... (4 Replies)
Discussion started by: evelibertine
4 Replies

5. UNIX for Dummies Questions & Answers

Writing a loop to merge multiple files by common column

I have 100 data files labelled 250.1.txt through 250.100.txt. The second column of the data files partially match (there is about %90 overlap). Each data file has 4 columns. I want the merge all these text files by the matching values in the second column. In the output, the first column should... (1 Reply)
Discussion started by: evelibertine
1 Replies

6. Shell Programming and Scripting

Writing a Perl Script that processes multiple files

I want to write a Perl script that manipulates multiple files. In the directory, I have files 250.*chr$.ped where * is from 1 to 1000 and $ is from 1-22 for a total of 22 x 10,000 = 22,000 files. I want to write a script that only manipulates files 250.1chr*.ped where * is from 1 to 22.... (10 Replies)
Discussion started by: evelibertine
10 Replies

7. UNIX for Dummies Questions & Answers

Writing a loop to manipulate a script and store it in multiple output files

I have a script where the the 9th line looks like this: $filename=sprintf("250.1chr%d.ped", $N); I want to modify this script 1000 times, changing 250.1chr%d.ped to 250.2chr%d.ped, 250.3chr%.ped.......and so on all the way to 250.1000chr%d.ped and store each output in files called ... (4 Replies)
Discussion started by: evelibertine
4 Replies

8. UNIX for Dummies Questions & Answers

Writing a for loop that processes multiple input files

I would like to write a for loop that does the following: I have a file called X.txt and other files called 1.txt,2.txt, .....,1000.txt. I want to substitute the 6th column of the file X.txt with 1.txt and store the output as X.1. Then I want to do the same with X.txt and 2.txt and store the... (1 Reply)
Discussion started by: evelibertine
1 Replies

9. UNIX for Dummies Questions & Answers

Reading and writing data to and from multiple files

Hi, I have several text files. One main file contains the detail data, other have some information to extract data from the main file, and some are empty files. Examples are shown below: The main file look like: MainFile.txt >Header1 data1...data1... >Header2 data2...data2... ... ...... (2 Replies)
Discussion started by: Fahmida
2 Replies

10. Shell Programming and Scripting

Writing out 2nd column into one file from multiple files

I have several files that are being generated every 20 minutes. Each file contains 2 columns. The 1st column is Text, 2nd column is Data. I would like to generate one single file from all these files as follows: One instance of 1st column Text, followed by 2nd column Data separated by... (5 Replies)
Discussion started by: subhap
5 Replies
Login or Register to Ask a Question