Intermittent "cp: cannot stat" error with nested loop


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Intermittent "cp: cannot stat" error with nested loop
# 1  
Old 06-14-2012
Intermittent "cp: cannot stat" error with nested loop

I have a bash script that has been running (on SUSE 9.3) dozens of times over the past couple of years without error. Recently it has been hitting intermittent “cp: cannot stat FILE: No such file or directory” errors.

The script has nested loops that continuously process files in a directory until the end condition is met. There are usually between 50 and 2000 files processed per execution. About 60% of the files processed hit a condition that requires the file to be copied to a new name so the new file can be processed in a future iteration. After the file is processed it is moved to a processed directory.

The error has not occurred more than one time per execution. Most times I can just re-run the script with the same data set and it works fine.

The strange thing is, according to the output log, the file it is complaining about did exist in the source directory and was moved to the processed directory. Another strange thing is the location of the “cannot stat” error in the output log seems to be random. Sometimes it appears in the middle of the output for the next file or even four or five files later.

Below is a distilled version of the code.


Code:
  Function_A() {
local ID=$1 local SUB=$2 if [ $CURRENT_COUNT -lt $MAX_COUNT ] then
nohup ChildProcess $ID $SUB > "$LOG" 2>&1 & let CURRENT_COUNT++ RC=0
else
echo "Cannot start another process right now" RC=1
fi return $RC
} # End Function_A ########################## # Main # # some unrelated detail here... # cd $DIR WaitingForChildrenToFinish=1 while [ $WaitingForChildrenToFinish -eq 1 ] do
# Process each file for FILE in `ls -tr ${MGR_ID}_*.msg 2>/dev/null` do
echo -e "\nProcessing file: \"$FILE\"" Function_A $MGR_ID $SUBJECT RC=$? if [ $RC -ne 0 ] then
RETRY="$DIR/${MGR_ID}_${RETRY}_${SUBJECT}.msg" cp -p $FILE $RETRY
fi # Finished processing this file so move it out mv $FILE $PROCESSED_DIR # # some unrelated detail here... # # End condition if [ $CO_END_FLAG -eq 1 ] then
WaitingForChildrenToFinish=0 break
fi
done # End for each file if [ $WaitingForChildrenToFinish -eq 1 ] then
echo "No files to process so sleep..." sleep 5
fi
done # End WaitingForChildrenToFinish Sample output looks like this: …. Processing file: "00wm4793_AAA_111.msg" Cannot start another process right now Processing file: "00wm4793_AAA_112.msg" cp: cannot stat `00wm4793_AAA_111.msg': No such file or directory Cannot start another process right now …

Any help will be appreciated. Thanks.


Moderator's Comments:
Mod Comment Please use code tags, thanks!

Last edited by zaxxon; 06-14-2012 at 09:51 AM.. Reason: code tags
# 2  
Old 06-14-2012
Might be here:
Code:
RETRY="$DIR/${MGR_ID}_${RETRY}_${SUBJECT}.msg"
cp -p $FILE $RETRY

On the right hand side of the assignment to RETRY, you have ${RETRY}. Perhaps this should be just the string RETRY? The more times the assignment executes the longer and more garbled the value of ${RETRY} becomes.
# 3  
Old 06-14-2012
Sorry, in my effort to cut out extraneous details I messed up that line.

It should be:
Code:
RETRY="$DIR/${MGR_ID}_${RETRY_WAIT}_${SUBJECT}.msg"
cp -p $FILE $RETRY

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

2. UNIX for Dummies Questions & Answers

Is "stat: illegal option -- -" an issue with hyphenated filename or flag problem?

Hi. I'm trying to install VMWare Workstation to run a virtual machine on my Mac OS, but running the bundle from bash(xterm) sh VMware-workstation-Full-11.0.0-2305329.x86_64.bundle (as suggested in install guide) comes up with error:stat: illegal option -- - usage: stat Digging... (5 Replies)
Discussion started by: defeated
5 Replies

3. AIX

Equalent of Linux "stat" in AIX

i would like to know the equivalent of stat -c %Y <file> command in AIX. i tried "istat" but its not giving the epoch time and also tried with perl perl -le'printf "%o", 07777 & (stat) for @ARGV' <file> it not also provding the timing . ... (3 Replies)
Discussion started by: expert
3 Replies

4. Shell Programming and Scripting

Bash (Ubuntu server): Syntax error: "|" unexpected in While-loop

Hello forum, I hope my problem is easy to solve for someone in here! My main task is to copy a large amount of imap-accounts from one server to another. There is a tool (Perl) called imapsync which does the job exellent. Unfortunately I'm only able to run it on one account at a time. After... (3 Replies)
Discussion started by: primaxx
3 Replies

5. Shell Programming and Scripting

Ignoring mv commands "cannot stat" error ?

So, my third thread here ^^ and still asking questions. Thanks for you patience and help, I really appreciated it ;) I currently use a shell script to move folders of songs from one to another location on my harddrive. I use something like this: sudo mv /var/mobile/Media/"My Music"/"Vasco... (3 Replies)
Discussion started by: pasc
3 Replies

6. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

7. Solaris

How to resolve error "INIT: Cannot stat /etc/inittab, errno: 2"

Hi All, I am getting an error message when I execute command “zlogin -C sunsrv4z5” on my root server. INIT: Cannot stat /etc/inittab, errno: 2 INIT: Cannot stat /etc/inittab, errno: 2 As per my analysis it seems that some files inside /etc folder are deleted. This server was... (14 Replies)
Discussion started by: surbhit4u
14 Replies

8. UNIX for Dummies Questions & Answers

what is a "find stat() error"

I'm searching for an oracle emtab file. I do a find / -name emtab -print and the first result gives me find: stat() error /apps/tomcat/jakarta-tomcat-5.5.9/bin/console.txt: I/O error Can someone explain what this error means? thanks, (2 Replies)
Discussion started by: orahi001
2 Replies

9. Shell Programming and Scripting

Avoid "++ requires lvalue" Error in Loop Calculation

Hi All, Please help me to perform sum of values in a loop, I am getting following error: "total=0++432907765772: ++ requires lvalue" where actual statement is as : total=$total+$amt where amt can have +ve or -ve values Thanks Sandeepb (3 Replies)
Discussion started by: sandeepb
3 Replies

10. Filesystems, Disks and Memory

Intermittent "slowdowns" on Solaris 2.6

We have a Sun Enterprise 450 running Solaris 2.6 that is giving us intermittent performance issues. These problems happen several times a day, usually lasting for a few minutes each time. sar shows that when this occurs, CPU idle time is 0% or close to it, and sys and wio are both high. iostat... (3 Replies)
Discussion started by: fiam
3 Replies
Login or Register to Ask a Question