Renaming directories stops resumption of write process


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Renaming directories stops resumption of write process
# 1  
Old 01-14-2016
Renaming directories stops resumption of write process

so lets say I have a process in cron that writes to a particular directory /var/tmp/EXAMPLEA

so, from time to time, say every couple of months, an upgrade is made. and here's how the upgrade works:

Code:
1. move the existing directory /var/tmp/EXAMPLEA somewhere else and name it differently...i.e. /var/tmp/EXAMPLEA-BKUP

2. create a new directory in its place, with the same name /var/tmp/EXAMPLEA

3. then, copy files that were in /var/tmp/EXAMPLEA-BKUP to /var/tmp/EXAMPLEA

Now, the problem is. after this upgrade is done, the process in cron does not resuming writing to the /var/tmp/EXAMPLEA directory even though everything is set up as it was before, with the write permissions and all.

can someone point in the direction of what the problem could be?

OS: Linux RedHat/CentOS/Ubuntu

Last edited by SkySmart; 01-14-2016 at 06:34 PM..
# 2  
Old 01-14-2016
Code?
My mind-reading glass-buble is currently out of order.
# 3  
Old 01-14-2016
Well there's a question!!!! I'm just thinking aloud and writing here, I don't know that I'm correct...............

I guess it depends on whether the cron job has the files open when the update runs. If so, it will (internally) be writing to specific inode numbers rather than specific files. In stage 3 when you 'copy' the files back to the original directory you create new inodes for those files. I'd be inclined to try renaming the files using 'mv' in stage 3 rather than 'cp'.

Let's see what other members suggest but that's my immediate thoughts.
This User Gave Thanks to hicksd8 For This Post:
# 4  
Old 01-14-2016
Quote:
Originally Posted by skysmart
Now, the problem is. after this upgrade is done, the process in cron does not resuming writing to the /var/tmp/EXAMPLEA directory even though everything is set up as it was before, with the write permissions and all.
That means you have currently running something.

Quote:
Originally Posted by skysmart
can someone point in the direction of what the problem could be?
Not without knowing where,what and how it is done.

Quote:
Originally Posted by hicksd8
I guess it depends on whether the cron job has the files open when the update runs. If so, it will (internally) be writing to specific inode numbers rather than specific files. In stage 3 when you 'copy' the files back to the original directory you create new inodes for those files. I'd be inclined to try renaming the files using 'mv' in stage 3 rather than 'cp'
One could make sure none of those files are open at the time the script is working with them.
Like, creating a lock file or something.

Otherwise... Need code!

Last edited by sea; 01-14-2016 at 07:22 PM..
This User Gave Thanks to sea For This Post:
# 5  
Old 01-14-2016
Are the files in /var/tmp/EXAMPLEA-BKUP growing after they were moved there from /var/tmp/EXAMPLEA?

If processes that were writing to those files had them open when they were moved and continue writing to the file descriptors that were used when they were opened, creating a new directory (with the old name or another name) will not cause those processes to magically start writing to a different file than they were writing to before. (And, note once a file descriptor is open, the file associated with that file descriptor is defined by the file system it is on and its i-node number; not by the pathname it had when it was opened.)
This User Gave Thanks to Don Cragun For This Post:
# 6  
Old 01-14-2016
Quote:
Originally Posted by Don Cragun
Are the files in /var/tmp/EXAMPLEA-BKUP growing after they were moved there from /var/tmp/EXAMPLEA?

If processes that were writing to those files had them open when they were moved and continue writing to the file descriptors that were used when they were opened, creating a new directory (with the old name or another name) will not cause those processes to magically start writing to a different file than they were writing to before. (And, note once a file descriptor is open, the file associated with that file descriptor is defined by the file system it is on and its i-node number; not by the pathname it had when it was opened.)

i presumed it had to be related to inodes.

and no, the files in /var/tmp/EXAMPLEA-BKUP do not continue growing when they're renamed. I think what I can try doing next time is:

1. Make sure the cron job is commented out or any application writing to the directories are shut down
2. And, as suggested by hicksd8, i can also use mv, instead of cp
# 7  
Old 01-14-2016
Quote:
Originally Posted by SkySmart
i presumed it had to be related to inodes.

and no, the files in /var/tmp/EXAMPLEA-BKUP do not continue growing when they're renamed. I think what I can try doing next time is:

1. Make sure the cron job is commented out or any application writing to the directories are shut down
2. And, as suggested by hicksd8, i can also use mv, instead of cp
Note that the following sequence of operations:
Code:
mv /var/tmp/EXAMPLEA /var/tmp/EXAMPLEA-BKUP
mkdir /var/tmp/EXAMPLEA
mv /var/tmp/EXAMPLEA-BKUP/* /var/tmp/EXAMPLEA

leaves you with an empty /var/tmp/EXAMPLEA-BKUP directory, and creates a period of time where files being written into /var/tmp/EXAMPLEA will be destroyed by replacing them with files being copied or moved from /var/tmp/EXAMPLEA-BKUP to /var/tmp/EXAMPLEA.

If you want to make a backup of files currently in a directory to a backup directory, it usually makes a lot more sense to do something more like:
Code:
rm -rf /var/tmp/EXAMPLEA-BKUP
mkdir /var/tmp/EXAMPLEA-BKUP
cp /var/tmp/EXAMPLEA/* /var/tmp/EXAMPLEA-BKUP

which never leaves /var/tmp/EXAMPLEA non-existent and never destroys any files that exist in that directory at the time the backup is created.
This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

IBM eServer X 335 stops boot process at Server splash screen

Hi there. I used to use linux a lot a number of years ago but it has been quite a while so ?I really consider myself a beginner again. I have an old IBM eServer X Series 335 machine and I want to get it back up and running for a number of reasons. I knew that one of the SCSI drives was bad and... (2 Replies)
Discussion started by: filch2
2 Replies

2. Solaris

Giving read write permission to user for specific directories and sub directories.

I have searched this quite a long time but couldn't find the right method for me to use. I need to assign read write permission to the user for specific directories and it's sub directories and files. I do not want to use ACL. This is for Solaris. Please help. (1 Reply)
Discussion started by: blinkingdan
1 Replies

3. Shell Programming and Scripting

Sendmail when a process stops

I have a process that crashes quite often. I'm unable to fix it since I'm not the developer and it is beyond my control to do so. However restarting the process will fix the issue which is fine. I would like to receive a notification via email that the process has failed so I can login and... (5 Replies)
Discussion started by: deaconf19
5 Replies

4. Shell Programming and Scripting

Renaming files in multiple directories

Hi I have the following file structure and I want to rename all the abc.jar files to abc_backup.jar rock@server:~/rakesh> ls -R .: test1 test2 test3 ./test1: abc.jar ./test2: abc.jar ./test3: abc.jar (2 Replies)
Discussion started by: rakeshkumar
2 Replies

5. Shell Programming and Scripting

Renaming bulk directories and subfiles

Hi, I have a directory with 100 subdirectories and each of these subdirectories has 1 file. Now I have to rename all these. The structure is "files directory has 100 SRR191639-SRR191718 subfolders and in each there is a file with the same name a subdirectory followed by .sra extension... (5 Replies)
Discussion started by: Diya123
5 Replies

6. Shell Programming and Scripting

renaming directories with shell script

Hi All after looking around the website and various other resources I become stuck. I'm trying to rename directories from Firstname Initial Lastname to lastname,_firstname_initial so far ive got for f in {./} do rename -n 'y/A-Z/a-z/' * rename -n 's/\ /_/g' * ... (2 Replies)
Discussion started by: harlequin
2 Replies

7. UNIX for Dummies Questions & Answers

Moving files out of multiple directories and renaming them in numerical order

Hi, I have 500 directories each with multiple data files inside them. The names are sort of random. For example, one directory has files named e_1.dat, e_5.dat, e_8.dat, etc. I need to move the files to a single directory and rename them all in numerical order, from 1.dat to 1000(or some... (1 Reply)
Discussion started by: renthead720
1 Replies

8. UNIX for Dummies Questions & Answers

Renaming files after their directory name in multiple sub directories

So I am not sure if this should go in the shell forum or in the beginners. It is my first time posting on these forums. I have a directory, main_dir lets say, with multiple sub directories (one_dir through onehundred_dir for example) and in each sub directory there is a test.txt. How would one... (2 Replies)
Discussion started by: robotsbite
2 Replies

9. UNIX for Dummies Questions & Answers

Batch Renaming: Change files' extensions in many sub-directories

Hi all - I'm trying to rename a large number of files all at once and need some help figuring out the command line syntax to do it. I've already done quite a bit of research with the rename and mv commands, but so far haven't found a solution that seems to work for me. So: The files exist... (10 Replies)
Discussion started by: dave920
10 Replies
Login or Register to Ask a Question