how to handle potential file contention Post: 302355359

Sponsored Content

Operating Systems AIX how to handle potential file contention Post 302355359 by philplasma on Tuesday 22nd of September 2009 11:24:32 AM

09-22-2009

Registered User

how to handle potential file contention

I need to change how a posting procedure currently works in order to improve load balancing but I am hitting a potential file contention problem that I was wondering if someone here could assist me with...

In a directory called FilePool I would have a bunch of files that are constantly coming in as follows:
File0001.GO
File0001.GOA
File0001.GOB
File0001.GOC
File0002.GO
File0002.GOA
File0002.GOB
File0002.GOC

The *.GO file for each batch of files is only written after the entire batch has been written, so the problem isn't that what is pulling these batches may conflict with what is writing them, but rather:

There are four processes that are each looking for the presence of a *.GO file in the FilePool directory. If one of these processes finds one, it takes the entire batch and moves it into that process' directory to process the batch.

These processes work quickly and it could be that two processes try to grab File0001.GO at the same time. Is there a way to
1 - have the first process lock File0001.GO if it isn't already locked and then move the whole batch
2 - have the process check if there is any other pid attached to that file first and then move it if there is none
3 - any other way that you would ensure that the file contention in this situation is handled?

Thank you for any assistance you can provide.

philplasma

View Public Profile for philplasma

Find all posts by philplasma

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

NFS file handle

How can I kill mount connection(NFS , made by automount) if remote filesystem is down? I tried: fuser -ku /auto /auto: umount /auto nfs umount: ERROR: /auto is busy If I try cd /auto - I get - /auto: Stale remote file handle. I know that reboot will help but I cannot reboot this...

2. Programming

deleting a file name by its handle

All, I am having three function 1.open 2.process 3.close. Open will open a file and process will process a file and close will close the file .. I can able to close the file by its filehandler.Is there is anyway that i can get the file name by filehandle and remove it after...

3. UNIX for Dummies Questions & Answers

Potential new user of Unix

Hi all, Complete and utter virgin Unix person here (I don't even have the OS yet) As I'm doing a "looking into it" kinda thing before I move from MS I hope my questions are not inappropriate. 1. Should I get some kind off anti virus software. I know Unix is pretty good for not getting them...

4. Shell Programming and Scripting

handle file by month

Hi, I want to write a shell script which handle an oracle alert log so that when it changes month, the script generates a new file with month in the filename. Thanks in advance. Leim

5. Solaris

Before I delete any file in Unix, How can I check no open file handle is pointing to that file?

I know how to check if any file has a unix process using a file by looking at 'lsof <fullpath/filename>' command. I think using lsof is very expensive. Also to make it accurate we need to inlcude fullpath of the file. Is there another command that can tell if a file has a truely active...

6. HP-UX

Potential file system contention on directory

We have an 8-processor Itanium system running HP-UX 11.23 connected to shared SAN discs. We have an application that creates files (about 10) in a specific directory. When the application terminates, these files are removed (unlink) and a few others are updated. The directory contains...

7. Cybersecurity

Attacking Potential of sh-scripts

Hey, I actually do have a question which seems rather easy for those you know more about this topic, since I am pretty new to bashscripting and don't know where it's limits are I have to ask you guys :) Imagine a system where all possible code execution methods (binary executables or...

8. Shell Programming and Scripting

Contention Identifier Script

Hi Experts, Need some help with a script which is definetly beyond my scripting skills. Here is flat file that I have with 4 Key Columns KEYCOLUMN1 KEYCOLUMN2 KEYCOLUMN3 KEYCOLUMN4 123ABC AEG MANCHESTER BIGBOX...

9. UNIX for Dummies Questions & Answers

Difference between handle to the thread HANDLE and thread identifier pthread_t

This question might be silly but its confusing me a bit: What is the difference between handle to the thread HANDLE and thread identifier pthread_t? ---------- Post updated at 01:52 PM ---------- Previous update was at 01:48 PM ---------- Sorry I saw details and HANDLE is in windows and...

LEARN ABOUT DEBIAN

ecaccess-queue-list

ECACCESS-QUEUE-LIST(1p) 				User Contributed Perl Documentation				   ECACCESS-QUEUE-LIST(1p)

NAME

       ecaccess-queue-list - List available queues

SYNOPSIS

       ecaccess-queue-list -version|-help|-manual

       ecaccess-queue-list [-debug] [queue-name]

DESCRIPTION

       Two different types of queues are in use with ECaccess:

       1 - The ECaccess batch queues, which correspond to one system at ECMWF with its specific batch environment, e.g. ecgate will be the
       ECaccess queue, which will redirect jobs to ecgate at ECMWF, running LoadLeveler. This ECaccess queue will be given as argument when
       submitting a batch job.

       2 - The batch queues (or classes) on the systems at ECMWF will be given in the batch job with #@ class = for LoadLeveler.

       The names of the ECaccess queues and associated batch queues at ECMWF can be seen with this command. When an ECaccess queue-name is
       specified then the details for this queue are displayed (including the list of classes).

       The first ECaccess batch queue in the list is the default queue when submitting a batch job with the ecaccess-job-submit command.

ARGUMENTS

       queue-name (optional)
	       The name of the ECaccess queue to retrieve the details.

OPTIONS

       -version
	       Display version number and exits.

       -help   Print a brief help message and exits.

       -manual Prints the manual page and exits.

       -debug  Display the SOAP messages exchanged.

EXAMPLES

       ecaccess-queue-list

       List all the ECaccess batch queues (or systems supporting job submission at ECMWF).

       ecaccess-queue-list ecgate

       List all the batch queues (or classes) on the ECMWF system ecgate.

SEE ALSO

       ecaccess.

perl v5.14.2							    2012-04-16						   ECACCESS-QUEUE-LIST(1p)