I need to change how a posting procedure currently works in order to improve load balancing but I am hitting a potential file contention problem that I was wondering if someone here could assist me with...
In a directory called FilePool I would have a bunch of files that are constantly coming in as follows:
File0001.GO
File0001.GOA
File0001.GOB
File0001.GOC
File0002.GO
File0002.GOA
File0002.GOB
File0002.GOC
The *.GO file for each batch of files is only written after the entire batch has been written, so the problem isn't that what is pulling these batches may conflict with what is writing them, but rather:
There are four processes that are each looking for the presence of a *.GO file in the FilePool directory. If one of these processes finds one, it takes the entire batch and moves it into that process' directory to process the batch.
These processes work quickly and it could be that two processes try to grab File0001.GO at the same time. Is there a way to
1 - have the first process lock File0001.GO if it isn't already locked and then move the whole batch
2 - have the process check if there is any other pid attached to that file first and then move it if there is none
3 - any other way that you would ensure that the file contention in this situation is handled?
for #1, can you temporarily rename the file?
for #2, you can use
, or
This will give you a list of process id's where the command args contain the filenames. 'man ps' for more info -- it's flavor-dependent, i.e., on solaris you might need /usr/ucb/ps instead (see this thread https://www.unix.com/shell-programmin...cates-o-p.html)
Can you have your processes write to an index file? Each proc could then look in this file for exclusions (i.e., filename in file, omit it from search,) write to the index when it claims ownership, and remove the entry when complete.
Last edited by varontron; 09-22-2009 at 01:48 PM..
Reason: more info
I'm going to try your renaming suggestion first - I've created a script in each of the two process directories to take the first *.GO file and mv it into the process folder. I am going to use at or cron to call these two scripts simultaneously and see what happens.
Presumably if you check the return value of mv for the rename then the one which attempts to run second will have failed and you can safely exit. Interestingly that avoids the race condition of trying to identically-named files at the same time.
This question might be silly but its confusing me a bit:
What is the difference between handle to the thread HANDLE and thread identifier pthread_t?
---------- Post updated at 01:52 PM ---------- Previous update was at 01:48 PM ----------
Sorry I saw details and HANDLE is in windows and... (0 Replies)
Hi Experts,
Need some help with a script which is definetly beyond my scripting skills.
Here is flat file that I have with 4 Key Columns
KEYCOLUMN1 KEYCOLUMN2 KEYCOLUMN3 KEYCOLUMN4
123ABC AEG MANCHESTER BIGBOX... (2 Replies)
Hey,
I actually do have a question which seems rather easy for those you know more about this topic, since I am pretty new to bashscripting and don't know where it's limits are I have to ask you guys :)
Imagine a system where all possible code execution methods (binary executables or... (15 Replies)
We have an 8-processor Itanium system running HP-UX 11.23 connected to shared SAN discs.
We have an application that creates files (about 10) in a specific directory. When the application terminates, these files are removed (unlink) and a few others are updated. The directory contains... (8 Replies)
I know how to check if any file has a unix process using a file by looking at 'lsof <fullpath/filename>' command.
I think using lsof is very expensive. Also to make it accurate we need to inlcude fullpath of the file.
Is there another command that can tell if a file has a truely active... (12 Replies)
Hi,
I want to write a shell script which handle an oracle alert log so that when it changes month, the script generates a new file with month in the filename.
Thanks in advance.
Leim (4 Replies)
Hi all,
Complete and utter virgin Unix person here (I don't even have the OS yet)
As I'm doing a "looking into it" kinda thing before I move from MS I hope my questions are not inappropriate.
1. Should I get some kind off anti virus software. I know Unix is pretty good for not getting them... (2 Replies)
All,
I am having three function
1.open
2.process
3.close.
Open will open a file and process will process a file and close will close the file ..
I can able to close the file by its filehandler.Is there is anyway that i can get the file name by filehandle and remove it after... (5 Replies)
How can I kill mount connection(NFS , made by automount) if remote filesystem is down?
I tried:
fuser -ku /auto
/auto:
umount /auto
nfs umount: ERROR: /auto is busy
If I try cd /auto - I get - /auto: Stale remote file handle.
I know that reboot will help but I cannot reboot this... (2 Replies)