Weird Error moving a file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Weird Error moving a file
# 1  
Old 04-30-2012
Weird Error moving a file

Hi,
We are running into very weird error on AIX. Target directory exists on NFS.

We have one script runs every 5 mins on our system. This script is running for 5 years for now without any issuess.

Followig is the snapshot of the script :

Code:
   stmt_cnt=$(ls -1 ${1}/${2}|wc -l) 
   if [ ${stmt_cnt} -ne 0 -a ${stmt_cnt} -lt 501 ]; then
     out_str="${out_str}\n$(date +"%Y%m%d%H%M%S"): Moving ${stmt_cnt} \"${2}\" statements from ${1} to ${3}\n"
     find ${1} -name "${2}" | sort -t"_" +7n +8n | xargs ksh -c 'mv $* '"${3}" foobar

Recently, mv command started failing only few occasions. Intrestingly, it is successful in the second run after failing in first run.

Following is the Error.

Code:
Usage: mv [-I] [ -d| -e] [-i | -f] [-E{force|ignore|warn}] [--] src target
   or: mv [-I] [-i | -f] [-E{force|ignore|warn}] [--] src1 ... srcN directory

Any guesses ? Appreciate pointers.

Last edited by Scott; 04-30-2012 at 04:52 PM.. Reason: Please use code tags
# 2  
Old 04-30-2012
Please post the exact parameters supplied to the script and mention what Shell this is.

Btw. What precisely is foobar in this context ?

It may have been running for 5 years but I for one cannot work out what this script is intended to do. Can you describe the process in words?
Please also post a complete script as this snippet reads as syntactically-incorrect nonsense.
# 3  
Old 04-30-2012
Wow! Didn't see that coming from a moderator.

How come it's syntactically-incorrect nonsense if it's working ?

Agreed, foobar is confusing here but works so syntax is not an issue

$1 = starting directory
$2 = pattern
$3 = destination

---------- Post updated at 05:02 PM ---------- Previous update was at 04:55 PM ----------

Question : Can system load be playing a role here ? Like output of find command not properly communicated to mv command and mv ends up with having empty string as src.

Code:
find ${1} -name "${2}" | sort -t"_" +7n +8n | xargs ksh -c 'mv $* '"${3}" foobar

Just speculating...

Last edited by methyl; 05-01-2012 at 10:27 AM.. Reason: please use code tags
# 4  
Old 05-01-2012
My guess is this:
If the total command line length is exceeded, then xargs calls the utility n times with fewer arguments. The first n-1 times with the maximum allowable line length and the last time with whatever remains. But since in this script the target directory is stuck on to the end, the total line length is exceeded for the mv command for the first n iterations, so these iterations of mv fail perhaps since the length is exceeded, or since the target directory is missing among it arguments. The last iteration gets passed what is left and thus had a lower line length and succeeds, so part of the files do get moved.

So the next time the script gets called, there are fewer files and this time there may be no error, because the total command line length does not get exceeded...

You could try lowering the command line length that xargs passes with the "-s" option.
# 5  
Old 05-01-2012
Thanks Scrutinizer. xargs wouldn't be an issue here. Interestingly, number of files to be processed are more second time around.

Example : 10 files came in, move failed all of them for the first run
Next Run 90 files came in.

All of 100 files got moved successfully for the second run. We have seen repetitions of the same behavior.

Last edited by methyl; 05-01-2012 at 06:58 PM.. Reason: correct spelling for readability
# 6  
Old 05-01-2012
Quote:
stmt_cnt=$(ls -1 ${1}/${2}|wc -l)
if [ ${stmt_cnt} -ne 0 -a ${stmt_cnt} -lt 501 ]; then
out_str="${out_str}\n$(date +"%Y%m%d%H%M%S"): Moving ${stmt_cnt} \"${2}\" statements from ${1} to ${3}\n"
find ${1} -name "${2}" | sort -t"_" +7n +8n | xargs ksh -c 'mv $* '"${3}" foobar
fi
The script needs urgent revision because it has some serious problems which may not produce syntax errors or error messages (?) but which cause alarming process anomalies due to command truncation in AIX. Have you counted the number of input files and compared with the number of output files?
I would suggest avoiding any unlimited xargs in AIX and executing comands one-by-one. Credit: Scrutinizer picked up on this.

Code:
stmt_cnt=$(ls -1 ${1}/${2}|wc -l) 
   if [ ${stmt_cnt} -ne 0 -a ${stmt_cnt} -lt 501 ]; then
     out_str="${out_str}\n$(date +"%Y%m%d%H%M%S"): Moving ${stmt_cnt} \"${2}\" statements from ${1} to ${3}\n"
     find ${1} -type f -name "${2}"|while read filename
     do
                mv "${filename}" "${3}"
     done
  fi

I have no idea what the sort is for and IMHO the ksh command line with the foobar is a botch to lose a surplus parameter when the command line is too long.
You clearly did not write this script but it is hard to justify that this script works despite the absence of reported syntax or other error messages.

Sorry to be so abrupt but that ksh line is a real problem.


Quote:
Agreed, foobar is confusing here but works so syntax is not an issue
Are you really sure? I'm not so sure.


Quote:
$1 = starting directory
$2 = pattern
$3 = destination
Let's assume that $3 is a pre-existing writeable directory. We don't know anything about $2, but that could be the subject of the next post.

Last edited by methyl; 05-01-2012 at 06:59 PM.. Reason: assorted corrections and addenda and footnote
# 7  
Old 05-23-2012
Quote:
Originally Posted by methyl
The script needs urgent revision because it has some serious problems which may not produce syntax errors or error messages (?) but which cause alarming process anomalies due to command truncation in AIX. Have you counted the number of input files and compared with the number of output files?
I would suggest avoiding any unlimited xargs in AIX and executing comands one-by-one. Credit: Scrutinizer picked up on this.

Yes, this script is working in production as intended.

Code:
stmt_cnt=$(ls -1 ${1}/${2}|wc -l) 
   if [ ${stmt_cnt} -ne 0 -a ${stmt_cnt} -lt 501 ]; then
     out_str="${out_str}\n$(date +"%Y%m%d%H%M%S"): Moving ${stmt_cnt} \"${2}\" statements from ${1} to ${3}\n"
     find ${1} -type f -name "${2}"|while read filename
     do
                mv "${filename}" "${3}"
     done
  fi

FYI.. -type f is not required

I have no idea what the sort is for and IMHO the ksh command line with the foobar is a botch to lose a surplus parameter when the command line is too long.
You clearly did not write this script but it is hard to justify that this script works despite the absence of reported syntax or other error messages.

Sorry to be so abrupt but that ksh line is a real problem.



Are you really sure? I'm not so sure.

Did you try out before asking ? As I mentioned , it's working for years in PRODUCTION how can you blame the syntax.

Let's assume that $3 is a pre-existing writeable directory. We don't know anything about $2, but that could be the subject of the next post.
Couldn't post the entire script due to copyright. Don't need tutorial on positional parameters. Anyway this discussion didn't head out into right direction.

I bet you understand the problem but have no clue. You are not alone.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Weird error after excutiong, completed fine though

Hi Guys - I'm getting the following error when I run my script: which: no Data_Export.sh in (.) However, my code completes with fine with a return code of 0. The above error message gets directed to my error file. Any ideas? Here is my script: #source... (1 Reply)
Discussion started by: SIMMS7400
1 Replies

2. Shell Programming and Scripting

Weird Error (: No such file or directory)

Hi Everyone, I am trying to make this script to use, to find out the DAHDI channel status. Every thing works fine, I even get proper results, however I have this weird error that comes along. Can someone please help me. Thanks a lot. #!/bin/bash # Color to set the test to when a channel is... (3 Replies)
Discussion started by: jeetz
3 Replies

3. Shell Programming and Scripting

awk weird error

Here is the awk code i wrote : if ; then gawk -v field_position="$field_position" -v field_length="$field_length" -v header="$header" -v trailer="$trailer" -v lr="$lr" '{ if(NR==1&&header=="1") { next } if(NR==lr&&trailer=="1") { next }... (1 Reply)
Discussion started by: ysvsr1
1 Replies

4. Shell Programming and Scripting

Weird Perl error using db2.

I have having a heck of a time figuring this out so any help is much appreciated. Here is the code where it seems to be dying, I bolded the part it is complaining about: $sth = $dbh->prepare( $query ) or die "error with query\n"; $sth->execute() or die "error executing query ...\n"; while(... (1 Reply)
Discussion started by: savigabi
1 Replies

5. Ubuntu

Need help with a weird sudo error.

I'm fairly new to unix and I was trying to change the name of my host and my user. I changed the name in /hostname using this: gksudo gedit /etc/hostname I then tried changing the name back but it still gave the same error: {env_reset,... (1 Reply)
Discussion started by: H3jck
1 Replies

6. Shell Programming and Scripting

Weird unbalanced quotes error

hi all, i am writing a wrapping script to burn subtitle into video file using transcode. I got this very weird error: code: inFile="movie.avi" subFile="sub.srt" outFile="movie_sub.avi" strExc="-i $inFile -x 'mplayer=-sub $subFile' -w $vidBR -o $outFile -y xvid" echo "transcode $strExc"... (2 Replies)
Discussion started by: tduccuong
2 Replies

7. Programming

C - advice how to catch some weird error

I have some unstable mistake in my program and out-of-idea how to catch it. I am looking for advice with a way to work it out! I have in a pretty complicated program (but one source file) set of int-counters - 15, if exactly. Lately, on final printout I have inpossible value (I am... (3 Replies)
Discussion started by: alex_5161
3 Replies

8. Web Development

weird 500 Internal server error

Hi All, I am seeking some help. While trying to access my website: EDITED (hosted on private server somewhere - don't want to publicize names) - I have a weird behaviour: I can always get to the site - but some applications get a 500 Internal error. If I use FireBug (mozilla addon) I can... (2 Replies)
Discussion started by: saariko
2 Replies

9. UNIX for Dummies Questions & Answers

File with a weird name 'q'

All, I have the following listing of files $ ls -lrt -rw-rw-rw- 1 user1 group1 335691 Aug 4 17:45 script1.sh -rw-rw-rw- 1 user1 group1 648 Aug 4 17:45 try,csv -rw-rw-rw- 1 user1 group1 1381275 Aug 4 17:45 test.txt -rw-rw-rw- 1 user1 group1 ... (7 Replies)
Discussion started by: rahulrathod
7 Replies

10. UNIX for Dummies Questions & Answers

Weird File

$ls -lrt -rw-r--r-- 1 rathodr users 1757 Jan 6 13:36 cleanup.archive.files.pl -rwxr-xr-x 1 rathodr users 20503 Jan 6 13:52 alarm.control.pl -rw-r--r-- 1 rathodr users 20503 Jan 9 04:52q The last file seems to be a weird file. I am not sure how was it created. Maybe... (5 Replies)
Discussion started by: rahulrathod
5 Replies
Login or Register to Ask a Question