Writing a loop to process multiple input files by a shell script


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Writing a loop to process multiple input files by a shell script
# 1  
Old 10-11-2011
Writing a loop to process multiple input files by a shell script

I have multiple input files that I want to manipulate using a shell script. The files are called 250.1 through 250.1000 but I only want the script to manipulate 250.300 through 250.1000. Before I was using the following script to manipulate the text files:

for i in 250.*; do
[ "$i" = 'X.txt' ] || awk 'NR==FNR{A[NR]=$6;next}{$6=A[FNR]}1' "$i" 'X.txt' > "X.${i%.txt}"
done

But the script above manipulates all scripts 1 through 1000 whereas I want it to specifically manipulate 300 through 1000. How can I change the script above to do that?
# 2  
Old 10-11-2011
What's your system?

What's your shell?

You've already been given a perl solution, what was wrong with it?
# 3  
Old 10-11-2011
My system is Ubuntu 11.04 and shell is bash, I believe.

Well that perl script worked but the output of the bash loop that I posted above was the input of the perl script. So I need to get this bash loop working first.
# 4  
Old 10-11-2011
Can this work for you:
Code:
for i in 250.[3-9]* 250.1000; do 
  [ "$i" = 'X.txt' ] || awk 'NR==FNR{A[NR]=$6;next}{$6=A[FNR]}1' "$i" 'X.txt' > "X.${i%.txt}"
done

# 5  
Old 10-11-2011
Quote:
Originally Posted by evelibertine
My system is Ubuntu 11.04 and shell is bash, I believe.
If you have BASH, you have for-loop:

Code:
for ((N=300; N<=1000; N++))
do
        FILENAME=250.$N

        ...
done

This would be better than for file in * since it's less likely to match other things by accident and isn't going to ever cram too many arguments for for to handle into anything.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Open Source

Help with writing Shell Script to automate process using multiple commands

Hello! Need help to write a Linux script that can be run from windows using command/Cygwin/any other way. I am new to scripting, actually i am trying to automate server health check like free disk space, memory along with few services status, if any services is not running then start services ,... (7 Replies)
Discussion started by: Sayed Ibrahim
7 Replies

2. Shell Programming and Scripting

Bash Question: HowTo Exit Script with User Input While Process is Running Mid-Loop?

Hi, I have written a script that allows me to repetitively play a music file $N times, which is specified through user input. However, if I want to exit the script before it has finished looping $N times, if I use CTRL+c, I have to CTRL+c however many times are left in order to complete the loop.... (9 Replies)
Discussion started by: hilltop_yodeler
9 Replies

3. Shell Programming and Scripting

FOR loop with multiple files as input and awk

Hi all , i want to pass multiple files as input to a for loop for i in file1 file2 file3 do some awk action < $i >> $i.out done but im getting error in that for loop is the way i use to pass files to awk using for correct and 2.we can directly pass multiple files to awk as... (7 Replies)
Discussion started by: zozoo
7 Replies

4. UNIX for Dummies Questions & Answers

Writing a for loop to manipulate multiple files

Hi, I have 1000 text files in a folder that are labeled data1.txt all the way to data1000.txt. I want to write a small script that manipulates the text files in this way: (1) cut the 2nd and 9th columns of the text files (2) sort by the numerical value in the 9th column (3) then save the rows... (3 Replies)
Discussion started by: evelibertine
3 Replies

5. UNIX for Dummies Questions & Answers

Using Shell Script To Loop Program Through Multiple Text Files

Hello, So I have approximately 300 files of raw data (.txt) files that I am using to perform statistical analysis. I have been able to construct a Fortran program that is able to perform my statistical analysis on a file by file basis. However, I now want to be able to loop program through... (19 Replies)
Discussion started by: Jimmyd24
19 Replies

6. UNIX for Dummies Questions & Answers

Writing a loop to merge multiple files by common column

I have 100 data files labelled 250.1.txt through 250.100.txt. The second column of the data files partially match (there is about %90 overlap). Each data file has 4 columns. I want the merge all these text files by the matching values in the second column. In the output, the first column should... (1 Reply)
Discussion started by: evelibertine
1 Replies

7. UNIX for Dummies Questions & Answers

Writing a loop to manipulate a script and store it in multiple output files

I have a script where the the 9th line looks like this: $filename=sprintf("250.1chr%d.ped", $N); I want to modify this script 1000 times, changing 250.1chr%d.ped to 250.2chr%d.ped, 250.3chr%.ped.......and so on all the way to 250.1000chr%d.ped and store each output in files called ... (4 Replies)
Discussion started by: evelibertine
4 Replies

8. UNIX for Dummies Questions & Answers

Writing a for loop that processes multiple input files

I would like to write a for loop that does the following: I have a file called X.txt and other files called 1.txt,2.txt, .....,1000.txt. I want to substitute the 6th column of the file X.txt with 1.txt and store the output as X.1. Then I want to do the same with X.txt and 2.txt and store the... (1 Reply)
Discussion started by: evelibertine
1 Replies

9. Shell Programming and Scripting

How to process multiple input files using Shell scripting

Hi, I would like to write a for loop that does the following: I have a file called X.txt and other files called 1.txt,2.txt, .....,1000.txt. I want to substitute the 6th column of the file X.txt with 1.txt and store the output as X.1. Then I want to do the same with X.txt and 2.txt and store... (0 Replies)
Discussion started by: evelibertine
0 Replies

10. Shell Programming and Scripting

How to Process input files from folder in shell script?

Hi, I want to process all input files available into folder (C:\ShellPrg\InputFile\) Input files are abc.CSV , XYZ.zip (zip of CSV file), PQR.gz (zip of CSV file). I want to check the extension of file, If its .zip/.gz then need to unzip the file as .CSV I want to parse line by line of... (2 Replies)
Discussion started by: Poonamol
2 Replies
Login or Register to Ask a Question