loop through numbered filenames


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting loop through numbered filenames
# 1  
Old 05-05-2009
loop through numbered filenames

Hi I'm very new to this script thing, so please be gentle.

I am trying to get a command - the mach2qtl command in the code below - to loop through a set of files.
Each command should take the same two .dat and .ped files, but the .mlinfo and .mlprob files with filenames including 'chrom1' should be run together and output to a file PC_full_1.txt

I have run the command fine by retyping it using the appropriate file names, but would like to run this as a script that runs through files 1 - 22.

When I run the script below I get a segmentational error.

What am I doing wrong?

Thanks,

Polly

Code:
 
#! /bin/bash
# program to run through each chromosome carrying out association using mach2qtl
 

# increment through chromosome numbers
i=0
while [ $i -lt 23 ]
        do
                i=$(( $i +1 ))
#list info and prob files 1 - 22
                infofile=chrom$i_step2.mlinfo
                probfile=chrom$i_step2.mlprob
# run mach2qtl using the above specified files
                mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile > PC_full_$i.txt

done
exit 0

# 2  
Old 05-05-2009
Hi
The option "--probfile" is correct or you have to use a single symbol "-"?

can you provide us the error message returned by the script ?
# 3  
Old 05-05-2009
error

Here is the list of errors, --probfile is correct, as I said this command works on its own, just not when in the script.

./assoc.sh: line 24: 31017 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31018 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31019 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31020 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31021 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31022 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31023 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31024 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31025 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31026 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31027 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31028 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31029 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31030 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31031 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31032 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31033 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31034 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31035 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31036 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31037 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31038 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31039 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
# 4  
Old 05-05-2009
I've realised the - while [ $i -lt 23 ] - should probably be -lt 22 instead.

However, I'm still getting the same fault message.
# 5  
Old 05-05-2009
Ok - I've put some 'echo's in so I could diagnose the problem


Code:
1
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4075 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
2
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4076 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
3
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4077 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
4
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4078 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
5
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4079 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
6
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4080 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
7
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4081 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
8
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4082 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
9
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4083 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
10
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4084 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
11
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4087 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
12
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4088 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
13
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4089 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
14
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4090 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
15
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4091 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
16
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4092 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
17
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4093 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
18
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4094 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
19
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4095 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
20
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4096 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
21
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4097 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
22
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4098 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt

so my incremental numbering is working put the value of 'i' is not being used in the filenames.

Code:
1
chrom.mlinfo
chrom.mlprob

should be:

Code:
1
chrom1.mlinfo
chrom1.mlprob

what am I doing wrong?

I've tried with and without double quotes (") around filenames.
# 6  
Old 05-05-2009
Ok solved it myself. I think.
I needed curly brackets around the variable i within the filename

Smilie
# 7  
Old 05-05-2009
The error message "segmentation fault" comes directly from the OS. Somehow it tries to execute the program "mach2qtl" and this program crashes. This is definitely not a problem coming out of your script, at least not directly.

A little hint: if you have a loop which behaves strangely try to "echo" out all the commands in question. This way you will see (instead of execute) what is to be execute:

Code:
i=0
while [ $i -lt 10 ] ; do
     command_in_question someparameter $i some_more_param
     (( i += 1 ))
done

Modify this to:
Code:
i=0
while [ $i -lt 10 ] ; do
     echo command_in_question someparameter $i some_more_param
     (( i += 1 ))
done

to see if what you think is executed is really executed. In loops more complicated than this example this can widely differ.

Another possibility is to insert "set -xv" at the start of the loop and "set +xv" at the end. This sends every line as it is interpreted to stderr. ("set -xv" turns this on, "set +xv" turns it off). You can monitor the script as it is executed like this:

Code:
script param1 param2 .... 2>&1 1>/dev/null | more

Lets talk about your script itself now:

Code:
infofile=chrom$i_step2.mlinfo

While not directly incorrect this could be misleading to the shell: suppose you have two variables, "$i" and "$i_". How should the shell find out which one you mean? If you want to expand a variable in the middle of a string write it this way:

Code:
infofile=chrom${i}_step2.mlinfo

This will remove all ambiguity. Further, make it a habit to ALWAYS QUOTE your strings, because a misplaced (or unexpected) space character could well lead to disaster, because it is a separator to the shell. Therefore always write

Code:
infofile="chrom${i}_step2.mlinfo"

and similarly for the other line in your script.

Btw., it seems that your variables "$infofile" and "$probfile" are not getting expanded, because otherwise there would be the contents of them in the error message.

I hope this helps.

bakunin
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

For/While Loop to Increment Filenames in a Script

Daily stupid question. I want to increment the file name everytime the script is run. So for example if the filename is manager.log and I run the script, I want the next sequence to be manager.log1. So to be clear I only want it to increment when the script is executed. So ./script... (10 Replies)
Discussion started by: metallica1973
10 Replies

2. Shell Programming and Scripting

Generate filenames in a loop

Hi all, I want to generate output files in a loop, run the same command on the same input file 1000 times and output in files with a new name each time, maybe a number appended to it. The output will be different each time as I`m sampling randomly from the input file. I want to do the... (3 Replies)
Discussion started by: newbie83
3 Replies

3. Shell Programming and Scripting

renaming numbered files

Hi there, I've got a set of files that are named as follows: image_N1_8letters.jpg image_N2_8letters.jpg ... image_N10_8letters.jpg image_N11_8letters.jpg .... image_N100_8letters.jpg image_N101_8letters.jpg with the "8letters" bit always consisting of 8 but always different... (3 Replies)
Discussion started by: kjartan
3 Replies

4. Shell Programming and Scripting

rename numbered files to numbered files with leading zeroes

Hi, I have some hundreds/thousands of files named logX.dat, where X can be any integer, and they are sequential, X ranges between 1 and any number: log1.dat log2.dat log3.dat log6.dat log10.dat ... log6000.dat I would like to rename them to scatter_params_0001.dat... (6 Replies)
Discussion started by: pau
6 Replies

5. Shell Programming and Scripting

For loop with dashes in filenames causing weird output

Hi again, What i'm trying to accomplish here is search a large directory for certain filesames, read from a txt file and looping through it. For instance, one of my target names from the loop file is: 1ad55f47-c342-496b-a46d-ba7de0f1b434 My loop is constructed thusly, run in a directory... (2 Replies)
Discussion started by: Karunamon
2 Replies

6. Shell Programming and Scripting

Listing even numbered files

Hi All, Could you please help in this case? Case: there's a directory 'CKMDB' in this directory, there are 30 files named in this manner 1.txt 2.txt 3.txt .... 30.txt Could you please guide me how to list only even numbered files like 2.txt... (5 Replies)
Discussion started by: xsam
5 Replies

7. Shell Programming and Scripting

Whitespace in filenames in for loop in bash script

I'm trying to search all .odt files in a directory for a string in the text of the file. I've found a bash script that works, except that it can't handle whitespace in the filenames. #!/bin/bash if ; then echo "Usage: searchodt searchterm" exit 1 fi for file in $(ls *.odt); do ... (4 Replies)
Discussion started by: triplemaya
4 Replies

8. UNIX for Dummies Questions & Answers

extracting and using date from filenames in a loop

HIya, Having a dumb day whilst writing an archive process in Shell want to extract from the filename the date and archive into tar files based on this, I don't want to use mtime as it may not be the actual file date. The files are -rw-rw---- 1 user admin 100 Aug 29 11:10... (2 Replies)
Discussion started by: badg3r
2 Replies

9. Shell Programming and Scripting

changing filenames in a directory to a number within a loop

hey guys. i'm new to shell scripting but not new to programming. i want to write a script that will take all the files in the current directory that end with a particular filetype and change all their names to a number in order. so, it would take all the jpg files and sort them in alphabetical... (30 Replies)
Discussion started by: visitorQ
30 Replies
Login or Register to Ask a Question