Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Writing a Perl Script that processes multiple files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Writing a Perl Script that processes multiple files
# 1  
Old 10-05-2011
Writing a Perl Script that processes multiple files
I want to write a Perl script that manipulates multiple files. In the directory, I have files 250.*chr$.ped where * is from 1 to 1000 and $ is from 1-22 for a total of 22 x 10,000 = 22,000 files.

I want to write a script that only manipulates files 250.1chr*.ped where * is from 1 to 22. Currently I am using
Code:
opendir(DIR,"/cchome/output")
#----------------------------------------------change dir
   || die "NO SUCH Directory: Images";
open(OUT,">>out.out")
   || die "cannot open output file";
while (@file = readdir(DIR) )
   {
      shift @file;
      shift @file;
      my $filename;
      foreach $_ (@file){
        $filename = $_;
        next unless ($filename =~ /250.1chr*.ped$/);

But it is not working. Could you help? Thanks!
Last edited by Franklin52; 10-06-2011 at 03:21 AM.. Reason: Please use code tags, thank you
# 2  
Old 10-05-2011
Those look like shell globs, perl uses regexes. so "250.1chr*.ped" tells it to match 250, "one of any character", 1ch, "zero or more r characters", "one of any character", ped, "end of line".

You're also assuming opendir() gives you the files in order. It's under no obligation to.

I can't tell what your filenames are with all that regex junk in them. Could you post one good, unmodified file name with no shell globs or symbol matching? Just the name.

Instead of using opendir, might it be easier to just check for the files you want?
This User Gave Thanks to Corona688 For This Post:
evelibertine
# 3  
Old 10-05-2011
Sure, the files name that I want to run the script on are:

250.1chr1.ped
250.1chr2.ped
250.1chr3.ped
.................
.................
.................
250.1chr21.ped
250.1chr22.ped
# 4  
Old 10-05-2011
Code:
for($N=1; $N<=22; $N++)
{
        $name=sprintf("/path/to/250.1chr%d.ped", $N);
        if(!open(FILE, "<${name}"))
        {
                print STDERR "Couldn't open ${name}\n";
                continue;
        }

        ...

        close(FILE);
}

This User Gave Thanks to Corona688 For This Post:
evelibertine
# 5  
Old 10-05-2011
Thanks but here are the error messages that I am getting:

Global symbol "$N" requires explicit package name at getout.pl line 14.
Global symbol "$N" requires explicit package name at getout.pl line 14.
Global symbol "$N" requires explicit package name at getout.pl line 14.
Global symbol "$filename" requires explicit package name at getout.pl line 16.
Global symbol "$N" requires explicit package name at getout.pl line 16.
Global symbol "$name" requires explicit package name at getout.pl line 17.
Global symbol "$name" requires explicit package name at getout.pl line 19.
syntax error at getout.pl line 20, near "continue"
syntax error at getout.pl line 23, near "..."
Execution of getout.pl aborted due to compilation errors
# 6  
Old 10-05-2011
I keep forgetting perl doesn't have continue. That's annoying, it has everything else you could think of...


Code:
for($N=1; $N<=22; $N++)
{
        $name=sprintf("/path/to/250.1chr%d.ped", $N);
        if(!open(FILE, "<${name}"))
        {
                print STDERR "Couldn't open ${name}\n";
        }
        else
        {
                close(FILE);
        }
}

As for the 'requires explicit package name', I don't get that at all. Depending on what options you've selected you may need to declare variables before using them but I can't see that from here. Please show the rest of your code.
This User Gave Thanks to Corona688 For This Post:
evelibertine
# 7  
Old 10-05-2011
This is my code:

Code:
#!/usr/local/bin/perl  -w
use Carp;
my @file;


for($N=1; $N<=22; $N++)
{
        $filename=sprintf("/cchome/perm/pedigree/250.1chr%.ped", $N);
        if(!open(FILE, "<${filename}"))
        {
                print STDERR "Couldn't open ${filename}\n";
        }
    else
    {
                close(FILE);
        }
}


        my $pheno_tmp = "";
        $pheno_tmp = $filename;

        if ($pheno_tmp =~ /(.+)chr/)
        {
                $pheno_tmp = $1;
        }
    else
    {
                $pheno_tmp = "Undef";
        }
    open (IN,"/cchome/perm/pedigree/output/$filename")   ||
die "cannot open input
file $filename";
        while (<IN>)
        {
    if ($_ =~ /Testing marker:/){chomp($_); my @marker = split(/\s+/,$_); my $marker = $marker[2]; print OUT $pheno_tmp, "\t", $marker,"\t$
        if ($_ =~ /probands/){chomp($_); my @pvalue = split(/\s+/,$_);
        my $Rsq = $pvalue[5]; my $pvalue = $pvalue[7]; print OUT "$Rsq\t$pvalue";print OUT "\n";}      #change the keywords;

        }
    close (IN);

close (OUT);

---------- Post updated at 01:14 PM ---------- Previous update was at 01:13 PM ----------

And these are the error messages that I'm getting:

Code:
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed
Couldn't open /cchome/perm/pedigree/250.1chr6451070ed

Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!
Last edited by vgersh99; 10-05-2011 at 03:19 PM.. Reason: code tags, please!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to run perl script on multiple files of two directories?

Hi I have 100 files under file A labled 1.txt 2.txt.....100.txt(made up name) I have 1 files under file B labled name.txt How can i run the same perl script on 100 files and file name.txt I want to run perl script.pl A/1.txt B/name.txt perl script.pl A/2.txt B/name.txt ....... perl... (3 Replies)
Discussion started by: grace_shen
3 Replies

2. UNIX for Dummies Questions & Answers

Writing a script to print the number of lines in multiple files

Hi I have 1000 files labelled data1.txt through data1000.txt. I want to write a script that prints out the number of lines in each txt file and outputs it in the following format: Column 1: number of data file (1 through 1000) Column 2: number of lines in the text file Thanks! (2 Replies)
Discussion started by: evelibertine
2 Replies

3. Shell Programming and Scripting

How can I do one liner import multiple custom .pm files in my perl script?

I am new for Perl I want to ask one question. I have around 50 custom packages which i am using in my Perl script. I want to import all .pm packages in my Perl script in an easy way. Right now i have to import each package individually. So Is there any way to do so?? Right Now i am doing like: ... (1 Reply)
Discussion started by: Navrattan Bansa
1 Replies

4. UNIX for Dummies Questions & Answers

Writing a loop to process multiple input files by a shell script

I have multiple input files that I want to manipulate using a shell script. The files are called 250.1 through 250.1000 but I only want the script to manipulate 250.300 through 250.1000. Before I was using the following script to manipulate the text files: for i in 250.*; do || awk... (4 Replies)
Discussion started by: evelibertine
4 Replies

5. UNIX for Dummies Questions & Answers

Writing a loop to manipulate a script and store it in multiple output files

I have a script where the the 9th line looks like this: $filename=sprintf("250.1chr%d.ped", $N); I want to modify this script 1000 times, changing 250.1chr%d.ped to 250.2chr%d.ped, 250.3chr%.ped.......and so on all the way to 250.1000chr%d.ped and store each output in files called ... (4 Replies)
Discussion started by: evelibertine
4 Replies

6. UNIX for Dummies Questions & Answers

Writing a for loop that processes multiple input files

I would like to write a for loop that does the following: I have a file called X.txt and other files called 1.txt,2.txt, .....,1000.txt. I want to substitute the 6th column of the file X.txt with 1.txt and store the output as X.1. Then I want to do the same with X.txt and 2.txt and store the... (1 Reply)
Discussion started by: evelibertine
1 Replies

7. Shell Programming and Scripting

Run perl script on files in multiple directories

Hi, I want to run a Perl script on multiple files, with same name ("Data.txt") but in different directories (eg : 2010_06_09_A/Data.txt, 2010_06_09_B/Data.txt). I know how to run this perl script on files in the same directory like: for $i in *.txt do perl myscript.pl $i > $i.new... (8 Replies)
Discussion started by: ad23
8 Replies

8. Shell Programming and Scripting

perl script on multiple files

I have a script that runs on one file (at a time). like this: $> perl myscript.pl filename > output How can I run it on >6000 files and have the output sent out into slightly modified file name $> perl myscript 6000files> output6000files.new extension Thanks in anticipation (4 Replies)
Discussion started by: aritakum
4 Replies

9. Shell Programming and Scripting

Multiple processes writing on the same file simultaneously

Hi All, I have encountered a problem,please help me. I have a script in which multiple processes are writing on to the same file . How should I stop this , I mean lock mechanism can be implemented or we can write the at different files and then concatenate the files. What would be a better... (1 Reply)
Discussion started by: Sayantan
1 Replies

10. Shell Programming and Scripting

Combining Multiple files in one in a perl script

All, I want to combine multiple files in one file. Something like what we do on the commad line as follows -> cat file1 file2 file3 > Main_File. Can something like this be done in a perl script very efficiently? Thanks, Rahul. (1 Reply)
Discussion started by: rahulrathod
1 Replies
Login or Register to Ask a Question