Run perl script on files in multiple directories


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Run perl script on files in multiple directories
# 1  
Old 06-24-2010
Run perl script on files in multiple directories

Hi,

I want to run a Perl script on multiple files, with same name ("Data.txt") but in different directories (eg : 2010_06_09_A/Data.txt, 2010_06_09_B/Data.txt).

I know how to run this perl script on files in the same directory like:

Code:
for $i in *.txt
do
perl myscript.pl $i > $i.new
done

But my filenames are same, and they are in different directories. How to go about this? This may be a very basic question, but I am just a beginner!!!

Thanks!
# 2  
Old 06-24-2010
Simply pass full/relative paths with directory name to your perl script. (This is assuming your perl script does not expect the file to be in the current working directory).

Code:
## Assuming the following directory structure:
## /abc/xyz/2010_06_09_A
## /abc/xyz/2010_06_09_B
## ...
## /abc/xyz/2010_06_09_Z

cd /abc/xyz
for i in */Data.txt
do
  perl myscript.pl $i > $i.new
done

If your perl script needs the file to be current working directory:

Code:
cd /abc/xyz
for i in */Data.txt
do
  dir=${i%/*}
  file=${i##*/}
  cd "$dir"
  perl myscript.pl ${file} > $i.new
  cd -
done


Last edited by a_programmer; 06-25-2010 at 12:13 PM.. Reason: typo correction: changed for $i to for i
This User Gave Thanks to a_programmer For This Post:
# 3  
Old 06-24-2010
Quote:
Originally Posted by ad23
...
I want to run a Perl script on multiple files, with same name ("Data.txt") but in different directories (eg : 2010_06_09_A/Data.txt, 2010_06_09_B/Data.txt).

I know how to run this perl script on files in the same directory like:

Code:
for $i in *.txt
do
perl myscript.pl $i > $i.new
done

But my filenames are same, and they are in different directories. How to go about this?
...
.... <Shell script snipped> ...
a_programmer's shell for loop is far better.

Perl globbing example follows -

You can iterate through all files named "Data.txt" in the Perl program itself, thereby avoiding the shell's for loop.

Example of your Perl script "myscript.pl" -

Code:
#!/usr/bin/perl -w
...
while ($file = glob("<your_directory>/*/Data.txt")) {
  # Now you have the full path in $file.
  # Process it the way you want.
  # You can even create and write to "$file.new" thereby 
  # avoiding the shell's redirection operator.
  ... your code here
}

HTH,
tyler_durden
This User Gave Thanks to durden_tyler For This Post:
# 4  
Old 06-25-2010
Question

I tried running the script according to a_programmer's code, but it is now giving me an error saying:
Code:
./myScript.sh: $i: is not an identifier

The following is my code:

Code:
#!/bin/tcsh
cd /abc/xyz
for $i in */Data.txt
do
    perl myScript.pl $i > $i.new
done

Am I overlooking something obvious in this?
Thanks again!
# 5  
Old 06-25-2010
Quote:
Originally Posted by ad23
I tried running the script according to a_programmer's code, but it is now giving me an error saying:
Code:
./myScript.sh: $i: is not an identifier

The following is my code:

Code:
#!/bin/tcsh
cd /abc/xyz
for $i in */Data.txt
do
    perl myScript.pl $i > $i.new
done

Am I overlooking something obvious in this?
Thanks again!
Remove the "$" sign and try
This User Gave Thanks to panyam For This Post:
# 6  
Old 06-25-2010
Quote:
Originally Posted by ad23
...
Code:
./myScript.sh: $i: is not an identifier

The following is my code:

Code:
#!/bin/tcsh
cd /abc/xyz
for $i in */Data.txt
do
    perl myScript.pl $i > $i.new
done

Am I overlooking something obvious in this?
...
Yes, you are. $i is a Perl variable; not a shell variable.

tyler_durden
# 7  
Old 06-25-2010
I am still not getting the desired output. Moreover when I print $i it is:

Code:
*/Data.txt

It is not taking the entire path for the file ?
What may be wrong?

---------- Post updated at 11:12 AM ---------- Previous update was at 11:11 AM ----------

When I run this :

Code:
cd abc/xyz

for i in */Data.txt
do
    #echo $i
    perl myScript.pl $i > $i.output
done

I get this error:
Code:
./yourScript.sh: line 10: */Data.txt.output: No such file or directory

Need your feedback??!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Run script on multiple files

Hi Guys, I've been having a look around to try and understand how i can do the below however havent come across anything that will work. Basically I have a parser script that I need to run across all files in a certain directory, I can do this one my by one on comand line however I... (1 Reply)
Discussion started by: mutley2202
1 Replies

2. Shell Programming and Scripting

Perl: script to work with files with the same name in different directories

Hi All, I would like to use a Perl (not Bash) script to work with multiple files of the same name in different directories (all in the same parent directory). I tried to create a loop to do so, but it isn't working. My code so far: while (defined(my $file = glob("./*/filename.txt")) or... (1 Reply)
Discussion started by: elgo4
1 Replies

3. Shell Programming and Scripting

Run script on multiple files

I have a script that I need to run on one file at a time. Unfortunately using for i in F* or cat F* is not possible. When I run the script using that, it jumbles the files and they are out of order. Here is the script: gawk '{count++; keyword = $1} END { for (k in count) {if (count == 2)... (18 Replies)
Discussion started by: newbie2010
18 Replies

4. Shell Programming and Scripting

Run one script on multiple files and print out multiple files.

How can I Run one script on multiple files and print out multiple files. FOR EXAMPLE i want to run script.pl on 100 files named 1.txt ....100.txt under same directory and print out corresponding file 1.gff ....100.gff.THANKS (4 Replies)
Discussion started by: grace_shen
4 Replies

5. UNIX for Dummies Questions & Answers

Run one script on multiple files and print out multiple files.

How can I run the following command on multiple files and print out the corresponding multiple files. perl script.pl genome.gff 1.txt > 1.gff However, there are multiples files of 1.txt, from 1----100.txt Thank you so much. No duplicate posting! Continue here. (0 Replies)
Discussion started by: grace_shen
0 Replies

6. Shell Programming and Scripting

How to run perl script on multiple files of two directories?

Hi I have 100 files under file A labled 1.txt 2.txt.....100.txt(made up name) I have 1 files under file B labled name.txt How can i run the same perl script on 100 files and file name.txt I want to run perl script.pl A/1.txt B/name.txt perl script.pl A/2.txt B/name.txt ....... perl... (3 Replies)
Discussion started by: grace_shen
3 Replies

7. Shell Programming and Scripting

Run perl script with multiple file arguments

Hello everyone, I have two types of files in a directory: *.txt *.info I have a perl script that uses these two files as arguments, and produces a result file: perl myScript.pl abc.txt abc.xml How can I run this script (in a "for" loop , looping through both types of files)... (4 Replies)
Discussion started by: ad23
4 Replies

8. Shell Programming and Scripting

perl script to check if empty files are created and delete them and run a shell script

I have a local linux machine in which the files are dumped by a remote ubuntu server. If the process in remote server has any problem then empty files are created in local machine. Is there any way using perl script to check if the empty files are being created and delete them and then run a shell... (2 Replies)
Discussion started by: hussa1n
2 Replies

9. UNIX for Dummies Questions & Answers

Foreach loop to run a perl script on multiple files

Hi, I have thousands of files in a directory that have the following 2 formats: 289620178.aln 289620179.aln 289620180.aln 289620183.aln 289620184.aln 289620185.aln 289620186.aln 289620187.aln 289620188.aln 289620189.aln 289620190.aln 289620192.aln.... and: alnCDS_1.fasta (1 Reply)
Discussion started by: greptastic
1 Replies

10. Shell Programming and Scripting

1 script or multiple scripts?? - check files, run jobs

Question for anyone that might be able to help: My objective is to eheck if a file (a source file) exists in a directory. If it does then, I'd like to call an application (Informatica ETL file...not necessary to know) to run a program which extracts data and loads it into multiple targets. ... (6 Replies)
Discussion started by: jnanasakti
6 Replies
Login or Register to Ask a Question