Run script on multiple files Post: 302883170

Sponsored Content

Top Forums Shell Programming and Scripting Run script on multiple files Post 302883170 by newbie2010 on Saturday 11th of January 2014 12:53:34 PM

01-11-2014

Registered User

Run script on multiple files

I have a script that I need to run on one file at a time. Unfortunately using for i in F* or cat F* is not possible. When I run the script using that, it jumbles the files and they are out of order. Here is the script:

Code:

gawk '{count[$2]++; keyword[$2] = $1}
END {
for (k in count)
{if (count[k] == 2) keyword[k] = "rearranging"
if (count[k] == 3) keyword[k] = "deleting"
if (count[k] > 3)  print k ": Volumes has more than 3 repeats" > /dev/stderr
else print keyword[k] " " k
}
}'

I have a number of files, all F* files that I need to apply it to, but I don't want the order of the files changed.

When I apply it to one file it works great, when I try to do for i in F* or while read file it jumbles the lines out of order.

Example of files:

Code:

cat F31
-==> /tmp/QCZ-RELAY-FILER_TASK-LIST <==
deleting   /vol/xyz_ent_0501
deleting   /vol/xyz_ent_0365
deleting   /vol/xyz_ent_0399
rearranging  /vol/xyz_ent_0363
rearranging  /vol/xyz_ent_0365
rearranging  /vol/xyz_ent_0393
rearranging  /vol/xyz_ent_0394
rearranging  /vol/xyz_ent_0399
rearranging  /vol/xyz_ent_0916
rearranging  /vol/xyz_ent_0494
rearranging  /vol/xyz_ent_0501
deleting   /vol/isis_root
deleting   /vol/xyz_ent_0393
deleting   /vol/xyz_ent_0494
deleting   /vol/xyz_ent_0394
deleting   /vol/xyz_ent_0363
deleting   /vol/xyz_ent_0916
deleting   /vol/xyz_ent_0916
rearranging  /vol/xyz_root

Code:

cat F20
--- SINGAPORE-QCX-RELAY-SERVER_TASK-LIST     
deleting   /vol/repository_vol10
deleting   /vol/repository_vol11
rearranging  /vol/repository_vol10
rearranging  /vol/repository_vol11
lab-nbu{528}$

I have also tried something like this:

Code:

for i in F*; do count=$((count+1)); cat F*  | sed -n "$count"p); /tmp/script_0108; done

But this halts. It seems that there is no way to use the regular for i in F* or anything similar, as it treats the files as a whole rather than working through them one by one

Is there a way to apply this script to one file at a time, pause and then have it go to the next file? For example, for the script to run on one file, then finish that iteration of its run, and go to the next file and then work on it?

newbie2010

View Public Profile for newbie2010

Find all posts by newbie2010

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

when I try to run rm on multiple files I have problem to delete files with space

Hello when I try to run rm on multiple files I have problem to delete files with space. I have this command : find . -name "*.cmd" | xargs \rm -f it doing the work fine but when it comes across files with spaces like : "my foo file.cmd" it refuse to delete it why?

2. Shell Programming and Scripting

1 script or multiple scripts?? - check files, run jobs

Question for anyone that might be able to help: My objective is to eheck if a file (a source file) exists in a directory. If it does then, I'd like to call an application (Informatica ETL file...not necessary to know) to run a program which extracts data and loads it into multiple targets. ...

3. UNIX for Dummies Questions & Answers

Foreach loop to run a perl script on multiple files

Hi, I have thousands of files in a directory that have the following 2 formats: 289620178.aln 289620179.aln 289620180.aln 289620183.aln 289620184.aln 289620185.aln 289620186.aln 289620187.aln 289620188.aln 289620189.aln 289620190.aln 289620192.aln.... and: alnCDS_1.fasta

4. Shell Programming and Scripting

How to run multiple awk files

I'm trying some thing like this. But not working It worked for bash files Now I want some thing like that along with multiple input files by redirecting their outputs as inputs of next command like below Could you guyz p0lz help me on this #!/usr/bin/awk -f BEGIN { } script1a.awk...

5. Shell Programming and Scripting

Run perl script on files in multiple directories

Hi, I want to run a Perl script on multiple files, with same name ("Data.txt") but in different directories (eg : 2010_06_09_A/Data.txt, 2010_06_09_B/Data.txt). I know how to run this perl script on files in the same directory like: for $i in *.txt do perl myscript.pl $i > $i.new...

6. Shell Programming and Scripting

Bash Scipting (New); Run multiple greps > multiple files

Hi everyone, I'm new to the forums, as you can probably tell... I'm also pretty new to scripting and writing any type of code. I needed to know exactly how I can grep for multiple strings, in files located in one directory, but I need each string to output to a separate file. So I'd...

7. Shell Programming and Scripting

How to run perl script on multiple files of two directories?

Hi I have 100 files under file A labled 1.txt 2.txt.....100.txt(made up name) I have 1 files under file B labled name.txt How can i run the same perl script on 100 files and file name.txt I want to run perl script.pl A/1.txt B/name.txt perl script.pl A/2.txt B/name.txt ....... perl...

8. UNIX for Dummies Questions & Answers

Run one script on multiple files and print out multiple files.

How can I run the following command on multiple files and print out the corresponding multiple files. perl script.pl genome.gff 1.txt > 1.gff However, there are multiples files of 1.txt, from 1----100.txt Thank you so much. No duplicate posting! Continue here.

9. Shell Programming and Scripting

Run one script on multiple files and print out multiple files.

How can I Run one script on multiple files and print out multiple files. FOR EXAMPLE i want to run script.pl on 100 files named 1.txt ....100.txt under same directory and print out corresponding file 1.gff ....100.gff.THANKS

10. UNIX for Dummies Questions & Answers