unusual problem with cp command


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers unusual problem with cp command
# 1  
Old 08-22-2012
unusual problem with cp command

I have made a simple script to zip a file then first copy it to a specific directory using cp command then move it to another directory. Files are getting generated at regular intervals in the dir. /one/two/three/four/. I have entry of my script in cron to run after every 2 min.
Code:
#!/bin/sh
date2=`date +%Y%m%d`
hstname=`hostname`
LOG_FNAME=Med_Gzip_"$date2"_"$hstname".txt
LOG_FILE_PATH="/one/two/three/ZIP_CDR_SCRIPT/$LOG_FNAME"
for FNAME in /one/two/three/four/MUM_NG2_*.cdr
do
   gzip "$FNAME"
   echo "Zipped File name : $FNAME " >> $LOG_FILE_PATH
   cp -p "$FNAME".gz /one/two/three/five/
   First_FNAME=`echo $FNAME |cut -d'.' -f1`
   Archive_FNAME="$First_FNAME"_Archived.cdr.gz
   mv "$FNAME".gz $Archive_FNAME
   mv $Archive_FNAME /one/two/three/five_archive
   date1=`date +%Y%m%d_%H%M%S`
   echo "ZIP FILE TIMESTAMP: $date1 " >> $LOG_FILE_PATH
   echo "********************" >> $LOG_FILE_PATH
done

Problem:
All zip files in the directory /one/two/three/five_archive are as expected but the files generated in the directory /one/two/three/five/ sometimes(randomly time interval in 4 to 10 days) have zero size and owner and group of the files are also gettnig changed to root. Permission also getting changed to 400 from 644.
No idea why this is happenning randomly, After every random number of days we are reported issue with file permission, owner, grop getting changed for few files.
Have anyone experienced same issue ever. Is there some issue with cp command. Any help would be highly appreciated.

Moderator's Comments:
Mod Comment Please view this code tag video for how to use code tags when posting code and data.

Last edited by vbe; 08-22-2012 at 01:10 PM.. Reason: code tags.. indent script
# 2  
Old 08-22-2012
WE know nothing about your system... (OS, version etc...)
There is no obvious issue with cp...
Are your manipulations in two distinct file systems? and more important:
How full is/are the file system(s) at that time? di you have a look at your system logs to see if there were anything for to me the only seen similar behaviour is when the file system is full...
# 3  
Old 08-22-2012
Vbe I will code tags from next time, thanks for reminding.

Below is OS details:
Linux pb3n1 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

Disk usage is around 75% to 85%

The directory where the script is copying the file is further read by another script which ftp the file from that directory . I have added chmod 666 before cp command but I am not sure how to tackle => sometimes file with zero size is getting copy to the destination directory while mv command output file have non-zero size.
# 4  
Old 08-22-2012
What I can see immediately from your code snippet
Code:
mv "$FNAME".gz $Archive_FNAME mv $Archive_FNAME /one/two/three/five_archive

is that $FNAME is quoted and $Archive_FNAME is not. This may lead to unexpected results, esp. for filenames with special chars in it. It would not explain the changes in files' owner, permission etc., of course. Did you try to run the script with the options -v and/or -x set so you can analyse every single step it does?

You say in your recent post that two processes are working simultaneously on the directory. Have you made sure that the operations are properly synchronized?
# 5  
Old 08-22-2012
Your last post was the thing to mention in your first post!
I have seen many tar backups fail because of out of sync between jobs in bad scheduling with cron and badly written scripts... In other words as suggest RudiC check again your sync if you are sure you are not running out of space, it is difficult to advise since we dont know what the job does at the same time...
# 6  
Old 08-23-2012
To expand a lot on vbe and RudiC's posts:

Does the first cron job ALWAYS complete in less than two minutes? If cron starts another job after two minutes and the previous one is still running, things will become awful rapidly.

Also: You are not checking either filesize or mtime of the .cdr file. How do you know if the .cdr file is ready to be archived? If mtime < 30 seconds old, the file may still be open and being written. The kernel syncs file data periodically, usually less than 1 minute intervals.

In any case you may be processing or trying to process incomplete or partially processed .cdr files. Note gzip does not rename the inputfile until after it completes, for example.

Also consider redirecting stderr in your cron entry:
Code:
2,4,6,8 * * * * /path/to/myscript.sh 2>>/path/to/errorlog.log

Then you can see what is going wrong. At least any errors.
This User Gave Thanks to jim mcnamara For This Post:
# 7  
Old 08-23-2012
jim mcnamara has got it.
You are sometimes picking up nascent files just after they have been created and before they contain data (hence the strange permissions). You could equally have been picking up part-written files which have the correct permissions. This situation tends to worsen with time once the directory file becomes fragmented.

We would suggest building a simple processing delay into the script by using the find command to only process files which are say at least one minute old (or more than the longest time that it takes to create a complete file - whichever is the greater). You can use find to check individual files.

The normal approach is to always use temporary file names when creating or copying a file, then rename the file to it's correct name when the process completes.
This User Gave Thanks to methyl For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. HP-UX

Unusual Behavior?

Our comp-operator has come across a peculiar ‘feature'. We have this directory where we save all the reports that were generated for a particular department for only one calendar year. Currently there are 45,869 files. When the operator tried to backup that drive it started to print a flie-listing... (3 Replies)
Discussion started by: vslewis
3 Replies

2. Shell Programming and Scripting

Unusual Problem

what is wrong with the below script: --------------------------------------------------------------------------------- #!/bin/bash echo "Setting JrePath..." grep -w "export JrePath" /etc/profile Export_Status=$? if echo "JrePath declared" elif echo "JrePath not declared" echo... (4 Replies)
Discussion started by: proactiveaditya
4 Replies

3. UNIX for Advanced & Expert Users

Unusual NFS mount problem on only ONE client: Red Hat WS Rel 3

This is an unusual situation where I have an NFS server currently serving out MULTIPLE clients over several variants of Linux and UNIX successfully (world permissions) except for a SINGLE client. Even the other Linux (SuSE) clients in the same room are mounting successfully with defaults without... (6 Replies)
Discussion started by: neelpert1
6 Replies

4. Programming

C Calender Help - Unusual error

I'm making a program that you input the month and year, and it creates a calender for that month of that year. This is my largest project yet, and I broke it up into several source files. cal.c #include "cal.h" #include <stdio.h> main() { int month, year; scanf("%d %d", &month,... (3 Replies)
Discussion started by: Octal
3 Replies

5. Shell Programming and Scripting

very unusual question about while

is there anyway to make while run a command faster than per second? timed=60 while do command sleep 1 done i need something that can run a script for me more than one time in one second. can someone help me out here? (3 Replies)
Discussion started by: Terrible
3 Replies

6. Solaris

pleaseee help with unusual crontab problem

Helllo folks... I tryed to edit crontab and I have this problem when I do crontab -l it shows my crontab correctly and if I do crontab -e I get this. baafh-99.03# baafh-99.03# crontab -e 1063 ? ? ? ? ? and that is all ...:( I have to type "q" and hit enter and I am back... (4 Replies)
Discussion started by: amon
4 Replies

7. Programming

unusual function refrences

I'm wrting a program which needs to get the following information of a sever by calling some lib fuctions or system calls, so can anybody help to tell me those function names or where I can find the description of them ? CPU usage Memory usage Load procs per min Swap usage Page I/O ... (11 Replies)
Discussion started by: xbjxbj
11 Replies

8. UNIX for Advanced & Expert Users

unusual function refrences

I'm wrting a program which needs to get the following information of a sever by calling some lib fuctions or system calls, so can anybody help to tell me those function names or where I can find the description of them ? CPU usage Memory usage Load procs per min Swap usage Page I/O Net I/O... (1 Reply)
Discussion started by: xbjxbj
1 Replies

9. UNIX for Dummies Questions & Answers

somewhat unusual top output problem

i'm a relative newbie to unix (i'm on OSX) and i have a specific problem i'm tripped up on: i'm piping the output of top (in log format) into an awk command which formats the information (and eventually will send it out continuously via udp/osc to another app). my problem is with what comes up... (4 Replies)
Discussion started by: ohhmyhead
4 Replies
Login or Register to Ask a Question