Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

My for loop decides to become an infinite loop?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting My for loop decides to become an infinite loop?
# 1  
Old 12-08-2011
My for loop decides to become an infinite loop?
Hi, I was debating if I should put this in the dummies or scripts section, I apologize in advance if I chose poorly.

Fairly new to Unix and BASH scripting but I thought I made it fairly well given my limited understanding. However, the output indicates that it's looping and I'm ending up with a stupid large number of files.

If you fine folks here could help me puzzle out a more efficient code, I would greatly appreciate it.
Code:
for dir in `ls -d1 /raid/r04/wconnor/Sites/*_30km`; do
#Find the list of directories my files are in, there are several 

    for res in modqkm mod375m modhkm mod750m mod1km; do
    #This is important to the ./spatbsq program 

        for file in `ls -S1 $dir | grep -v .hdr`; do
        #Each directory contains a list of files, [file] and [file].hdr. this is to just grab all the non-.hdr files

        ./spatbsq -DS -F-9999 -B1000x6x1000 hyp2${res}.par ${dir}/${file} ${dir}/${file}.${res}
        cp ${dir}/${file}.hdr ${dir}/${file}.${res}.hdr
        done
        #runs the spatbsq on the list above and then spits out the same filename with the ${res} on the end for both [file] and [file].hdr
    done
done

He's the output from a folder with only one [file] and associated hdr. I don't know how to get the forum to make a scroll window so sorry for the long post. I've sorted it by date to try and help me understand what the script is doing. I'm pretty sure the problem arises because of the "for file in `ls -S1 $dir | grep -v .hdr`" but I'm at a loss of how I could rewrite this so it would only read it once. I've had no luck trying to set it as a variable but that may just be because I don't know how to do it properly. Perhaps outputting the ls command to a file then doing a cat as the argument in the for loop?

Code:
total 375940
-rw-rw-r-- 1 wconnor proj      852 Dec  5 13:44 lndsr_L71116034_03420090406_GDK_30km.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  5 13:44 lndsr_L71116034_03420090406_GDK_30km
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:19 lndsr_L71116034_03420090406_GDK_30km.modqkm
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:19 lndsr_L71116034_03420090406_GDK_30km.modqkm.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:20 lndsr_L71116034_03420090406_GDK_30km.mod375m
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:20 lndsr_L71116034_03420090406_GDK_30km.mod375m.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:21 lndsr_L71116034_03420090406_GDK_30km.modqkm.mod375m
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:21 lndsr_L71116034_03420090406_GDK_30km.modqkm.mod375m.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:22 lndsr_L71116034_03420090406_GDK_30km.modhkm
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:22 lndsr_L71116034_03420090406_GDK_30km.modhkm.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:24 lndsr_L71116034_03420090406_GDK_30km.mod375m.modhkm
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:24 lndsr_L71116034_03420090406_GDK_30km.mod375m.modhkm.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:25 lndsr_L71116034_03420090406_GDK_30km.modqkm.modhkm
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:25 lndsr_L71116034_03420090406_GDK_30km.modqkm.modhkm.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:26 lndsr_L71116034_03420090406_GDK_30km.modqkm.mod375m.modhkm
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:26 lndsr_L71116034_03420090406_GDK_30km.modqkm.mod375m.modhkm.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:30 lndsr_L71116034_03420090406_GDK_30km.mod750m
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:30 lndsr_L71116034_03420090406_GDK_30km.mod750m.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:32 lndsr_L71116034_03420090406_GDK_30km.mod375m.mod750m
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:32 lndsr_L71116034_03420090406_GDK_30km.mod375m.mod750m.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:34 lndsr_L71116034_03420090406_GDK_30km.mod375m.modhkm.mod750m
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:34 lndsr_L71116034_03420090406_GDK_30km.mod375m.modhkm.mod750m.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:36 lndsr_L71116034_03420090406_GDK_30km.modhkm.mod750m
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:36 lndsr_L71116034_03420090406_GDK_30km.modhkm.mod750m.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:39 lndsr_L71116034_03420090406_GDK_30km.modqkm.mod750m
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:39 lndsr_L71116034_03420090406_GDK_30km.modqkm.mod750m.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:41 lndsr_L71116034_03420090406_GDK_30km.modqkm.mod375m.mod750m
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:41 lndsr_L71116034_03420090406_GDK_30km.modqkm.mod375m.mod750m.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:43 lndsr_L71116034_03420090406_GDK_30km.modqkm.mod375m.modhkm.mod750m
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:43 lndsr_L71116034_03420090406_GDK_30km.modqkm.mod375m.modhkm.mod750m.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:44 lndsr_L71116034_03420090406_GDK_30km.modqkm.modhkm.mod750m
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:44 lndsr_L71116034_03420090406_GDK_30km.modqkm.modhkm.mod750m.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:51 lndsr_L71116034_03420090406_GDK_30km.mod1km
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:51 lndsr_L71116034_03420090406_GDK_30km.mod1km.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:56 lndsr_L71116034_03420090406_GDK_30km.mod375m.mod1km
-rw-rw-r-- 1 wconnor proj      852 Dec  6 17:56 lndsr_L71116034_03420090406_GDK_30km.mod375m.mod1km.hdr
-rw-rw-r-- 1 wconnor proj 12000000 Dec  6 17:59 lndsr_L71116034_03420090406_GDK_30km.mod375m.mod750m.mod1km

..  It goes on further but you get the idea

# 2  
Old 12-08-2011
That's a dangerous use of backticks, useless use of ls *, and useless use of cat. You can't cram arbitrary sizes of things into backticks and shouldn't try. Either use pipes or temp files.

I can't quite understand what you want it to be doing from your malfunctioning program. Could you show what files it's supposed to be using, what files it's supposed to get out, and which files create what?

I'd start with:

Code:
# Find all files inside /raid/r04/wconnor/Sites/*_30km that aren't named '*.hdr', save to temp file.
# The temp file won't change later, so 'find' can't feed your own files created files back into your script.
find /raid/r04/wconnor/Sites/*_30km -type f '!' -name '*.hdr' > /tmp/$$

# Read lines from temp file
while read FILE
do
        ...
done < /tmp/$$

rm -f /tmp/$$

...but what you want done with the files isn't too clear.
This User Gave Thanks to Corona688 For This Post:
gotreef
# 3  
Old 12-08-2011
@Corona

This is exactly what I was hoping for. You're telling me that how I think about scripting is garbage and why it's bad. Although telling me how poorly I've written my script, I feel that this better belongs in the noob section.

What I'm trying to do is a bit obtuse so please forgive me if it's poorly explained:
I have a series of satellite images from various locations which are named along the lines of:
Image_date_location
Image_date_location.hdr

These images are of a pixel size (30m) but I'm going to fuzz them out to simulate larger pixel footprints, (250m, 375m, 500m, and 1km); this is accomplished with spatbsq and specifying the parameter $res.

After the 'spatbsq' is done, it should output:
Image_date_location.250m
Image_date_location.250m.hdr
and so on for all I would in theory end up with 4 new files for every original file, one for each resolution.

It's good to know that I shouldn't use backticks in these cases
Could you (or someone else) possibly explain to me what 'done < /tmp/$$' does and why $$ is used instead of a file name?
# 4  
Old 12-08-2011
$$ is just the PID of the shell, a decimal number. It's used as a unique temporary file name. Getting it from mktemp is more technically correct but some people don't have that.

Thanks for the explanation, but I'm still missing a lot of pieces. These 'par' files you seem to be using with every file to create different files for instance. Is running the utility x times for one set of files what's supposed to be happening, each time with a different .par file?

And what's the cp for? Why copy the same file into a new name over and over? Is this ./spatbsq command altering the original files each time you run it?

How about a hypothetical example. You have only one set of files, file and file.hdr. What are the exact commands you run to convert them?
This User Gave Thanks to Corona688 For This Post:
gotreef
# 5  
Old 12-08-2011
Thanks again for the response, it's greatly appreciated. My .par files are located in the same directory as sparbsq and the script. Without getting into the details of what they contain, there are 5 of them listed as:
hyp2modqkm.par, hyp2mod375m.par, hyp2modhkm.par, hyp2mod750m.par, hyp2mod1km.par

I need to run spatbsq on every image file 5 times, each time with a different .par as an input.

Hope hypothetical example v2.0 will be more clear about what I need to do. My files are:
/newyork/brooklynbridge
/newyork/brooklynbridge.hdr

I would run it roughly as follows:
./spatbsq [.par] [input] [output]
./spatbsq hyp2qkm.par /newyork/brooklynbridge /newyork/brooklynbridge.qkm
./spatbsq hyp2375m.par /newyork/brooklynbridge /newyork/brooklynbridge.375m
./spatbsq hyp2hkm.par /newyork/brooklynbridge /newyork/brooklynbridge.hkm
./spatbsq hyp2750m.par /newyork/brooklynbridge /newyork/brooklynbridge.750m
./spatbsq hyp21km.par /newyork/brooklynbridge /newyork/brooklynbridge.1km

so in regards to renaming, all I do is essentially tack on the suffix of the .par to my output file. The cp is because each of these outputs need a .hdr file of the same name. The .hdr is simply spatial information for the photo and since the brooklyn bridge doesn't really shift around, I just copy the original .hdr to one with the appropriate $res suffix.

---------- Post updated at 02:06 PM ---------- Previous update was at 01:29 PM ----------

It looks like your little script snippit solved all my problems plus I feel like I've learned a lot as well, for which I'm greatful.

One quick thing though. When you have the "while read FILE", how does it know that the FILE is the /tmp/$$ file?
# 6  
Old 12-09-2011
Quote:
Originally Posted by gotreef
It looks like your little script snippit solved all my problems plus I feel like I've learned a lot as well, for which I'm greatful.

One quick thing though. When you have the "while read FILE", how does it know that the FILE is the /tmp/$$ file?
Glad it worked for you.

It doesn't -- it just reads from standard input. But the </tmp/$$ tells it what "standard input" is supposed to be, at the bottom of the loop.

Imagine it like this:
Code:
read FILE </tmp/$$

...except that there's an entire loop involved:
Code:
while read FILE
do
...
done </tmp/$$

...in which the file gets opened once and put into standard input for read and anything else inside the loop to read as it pleases.

You can do that, in shell -- redirect the input and/or output of entire loops and statements:

Code:
# all 5 echo statements go into the same file
for N in 1 2 3 4 5
do
        echo $N
done > output

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Infinite loop query

I have a script script.shwhich is scheduled to run at 11 AM everyday. # script.sh Code: ./scb_script.sh & unfortunately scb_script.sh is running today in infinite loop as respective files are not available. My question, when script.sh starts running tomorrow, will the old process be... (1 Reply)
Discussion started by: JSKOBS
1 Replies

2. Shell Programming and Scripting

How to stop infinite loop

Im unable to stop the below infinite loop (bash script). Can someone tell me why this isnt responding to signals eg: ctrl+c (SIGINT) or ctrl+z c=0 test_loop() { c=$(($c+1)) echo "count value is : $c " sleep 1 test_loop } Im using: SunOS 5.10 PS: If run this as... (13 Replies)
Discussion started by: Arun_Linux
13 Replies

3. Homework & Coursework Questions

Help with infinite loop problem

1. The problem statement, all variables and given/known data: My problem is an infinite loop when i press any other key other then Y or y in the while loop. what i want it to do is return to the normal script outside of it if pressing N or n or keep asking the same question if its any other... (4 Replies)
Discussion started by: Ren_kun
4 Replies

4. Shell Programming and Scripting

Call a infinite loop

Hi All, I need to run an infinite loop. requirement below: function1 --> creates a file file1 function2 ---> need to call if the file creates i am running these both function via a script --> script.sh i need to run the function1 first and if the file file1 creates then need to run the... (3 Replies)
Discussion started by: satyaranjon
3 Replies

5. UNIX for Advanced & Expert Users

Procmail and infinite loop

I wanted to copy (not forward but copy) all incoming email to another address of mine. It worked, but now I encountered an infinite loop problem: When the second address doesn't like the content and bounces the message back, the bounce message will be sent back and forth. So, what I have in... (1 Reply)
Discussion started by: distill
1 Replies

6. Shell Programming and Scripting

Infinite while loop

what is the difference between while:,while true and while false? (6 Replies)
Discussion started by: proactiveaditya
6 Replies

7. Shell Programming and Scripting

infinite while do loop problem

hi all, this is how my scrip looks like #!/bin/sh bindir='/opt/apps/script/bin' datadir='/opt/apps/script/data' dir='/opt/apps/script' while : ; do ls -1rt /opt/apps/script/data/check.txt*|tail -1 > /dev/null 2>&1 if ;then chmod +rwx $bindir/dummy2.sh ... (8 Replies)
Discussion started by: tententen
8 Replies

8. Programming

Is my code in an Infinite Loop?

Production C code compiled without the dash-g option is running, and seems to be in an infinite loop. Is there a way to tell? Is there a diagnostic tool that will report what objects or what lines of code or even what functions are being executed? Or is my best option to kill it with a dump? ... (5 Replies)
Discussion started by: marcus121
5 Replies

9. Shell Programming and Scripting

Infinite loop not looping

Hi guys, I'm having a problem getting my infinite loop to loop. It simply reads in the users choice form the menu, executes the corresponding case statement and quits instead of looping back to the main menu again. I have a feeling it might be something with my if then statements within the case... (2 Replies)
Discussion started by: hootdocta5
2 Replies

10. Solaris

ls command in infinite Loop

Hi, whenever I am giving a 'ls' command system is going into infinite loop displaying the current home directory. There is no separate shell script/file with ls name anywhere in the system. I am using Solaris 10. Any help / guidance in solving this problem is highly appreciated. ... (3 Replies)
Discussion started by: umakant
3 Replies
Login or Register to Ask a Question