Sponsored Content
Top Forums Shell Programming and Scripting Multiple process write to same log file at the same time Post 302456990 by citaylor on Monday 27th of September 2010 04:56:06 AM
Old 09-27-2010
The example will not cause a race condition, at worst it will corrupt the log file. However, If the processes write the data in single write statements (most do, either by using single "write" API calls or by using streamed file I/O which will buffer output), and each log line is less than 4kb (streamed I/O is), and the log file is local, then most modern UNIX/Linux hosts will not intermingle bytes within separate log lines. This is only a rule of thumb however, so I would not rely on this to work 100% across different O/S.

From your example in the first post it seems you launching the processes from the shell and are separate processes, and therefore you cannot use mutex's as matrixmadhan has suggested. Is this correct, or are you in control of the code, and could you implement threads (as mutexes are very useful and would solve this issue). Otherwise, Im afraid that I am in disagreement with matrixmadhan's suggestion that lock files are outdated - they are still a very useful tool which most daemons and other applications rely on (for example the whole System V "init" script architecture on most UNIX/Linux implementations still use this mechanism in creating "pid" files in /var/run, etc).

So I would say that if you are creating a simple application or script that calls multiple processes and isnt 100% mission critical/bullet proof, then "lockfile" would probably be a useful tool for you. If, however, you are developing a mission critical service that has to 100% guarantee data integrity, that you should look into more advanced synchronization techniques and/or data repository services.

I hope this helps.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Script to Extract time from log files and write to a excel

Can someone help me with writing a unix script for following requirement 1) I have a log file in which we have start time and end time (format: hh:mm:ss) Example: starting script on Thu Jun 5 20:50:52 Thu Jun 5 21:55:33 - Script Completed 2) I want to extract start time and end time of... (0 Replies)
Discussion started by: santosham
0 Replies

2. Shell Programming and Scripting

Script to Extract time from log files and write to a excel

Can someone help me with writing a unix script for following requirement 1) I have a log file in which we have start time and end time (format: hh:mm:ss) Example: starting script on Thu Jun 5 20:50:52 Thu Jun 5 21:55:33 - Script Completed 2) I want to extract start time and end time of... (0 Replies)
Discussion started by: santosham
0 Replies

3. UNIX for Dummies Questions & Answers

Script to Extract time from log files and write to a excel

Can someone help me with writing a unix script for following requirement 1) I have a log file in which we have start time and end time (format: hh:mm:ss) Example: starting script on Thu Jun 5 20:50:52 --------- Thu Jun 5 21:55:33 - Script Completed 2) I want to extract... (4 Replies)
Discussion started by: santosham
4 Replies

4. Shell Programming and Scripting

how to run multiple process at the same time

Hello guys, Look what im doing: I need to run a process from a SERVER1 to SERVER2, SERVER3 and SERVER4. The shell of the process is in each SERVER (2 to 4) So from SERVER1 i do: for i in SERVER2 SERVER3 SERVER4 do rsh $i ' ./process.sh ' done The problem is: each process.sh... (2 Replies)
Discussion started by: lestat_ecuador
2 Replies

5. Shell Programming and Scripting

Write process output to a file

When I run <ls -l> to get a list of all the files, I want the displayed result to be written to a text file. Is there a way to do that? (1 Reply)
Discussion started by: kn.naresh
1 Replies

6. UNIX for Advanced & Expert Users

when a process fails to write to /dev/log

Hi , when a process fails to write to /dev/log ? (1 Reply)
Discussion started by: Gopi Krishna P
1 Replies

7. UNIX for Advanced & Expert Users

Multiple processes write to log file at the same time

If we have 3 process to write to same log file at the same time like below. will it cause the data outdated because the multiple process writing same time? It this a safe way to keep the log for multiple process? p1 >> test.log &; p2 >> test.log &; p3 >> test.log & Thanks, (1 Reply)
Discussion started by: casttree
1 Replies

8. Shell Programming and Scripting

Batch write multiple usb in same time

Hi there, I would write a bash script to format then create a range of folder (folder name is number) to a usb stick. for numbers in $(seq -w 001 999) do pause "Press any key to start" mkfs.vfat -F32 /dev/sdc1 mount /dev/sdc1 /media/usb mkdir /media/$numbers umount /dev/sdc1... (8 Replies)
Discussion started by: wayneliao38
8 Replies

9. Shell Programming and Scripting

Ksh: How to write the log file simultaneously when .sql file executes by UNIX process

Hello Team- we would like to implement an approach which has to write the log file simultaneously when .sql file is executing by Unix process. At present,it is writing the log file once the process is completed. I've tested the current process with the below approaches and none of them... (1 Reply)
Discussion started by: Hima_B
1 Replies

10. Shell Programming and Scripting

Subprocess.popen() should write to log without waiting for process to complete

Here is my test code process = sp.Popen( + , bufsize=1, universal_newlines=True, stdout=sp.PIPE, stderr=sp.STDOUT, cwd=src_home) output, _ =... (2 Replies)
Discussion started by: ezee
2 Replies
mutex(5)                                                Standards, Environments, and Macros                                               mutex(5)

NAME
mutex - concepts relating to mutual exclusion locks DESCRIPTION
Mutual exclusion locks (mutexes) prevent multiple threads from simultaneously executing critical sections of code which access shared data (that is, mutexes are used to serialize the execution of threads). All mutexes must be global. A successful call to acquire a mutex will cause another thread that is also trying to lock the same mutex to block until the owner thread unlocks the mutex. Mutexes can synchronize threads within the same process or in other processes. Mutexes can be used to synchronize threads between processes if the mutexes are allocated in writable memory and shared among the cooperating processes (see mmap(2)), and have been initialized for this task. The following table lists mutex functions and the actions they perform. +-----------------------+-----------------------------------+ | FUNCTION | ACTION | |mutex_init | Initialize a mutex. | |mutex_destroy | Destroy a mutex. | |mutex_lock | Lock a mutex. | |mutex_trylock | Attempt to lock a mutex. | |mutex_unlock | Unlock a mutex. | |pthread_mutex_init | Initialize a mutex. | |pthread_mutex_destroy | Destroy a mutex. | |pthread_mutex_lock | Lock a mutex. | |pthread_mutex_trylock | Attempt to lock a mutex. | |pthread_mutex_unlock | Unlock a mutex. | +-----------------------+-----------------------------------+ Initialization Mutexes are either intra-process or inter-process, depending upon the argument passed implicitly or explicitly to the initialization of that mutex. A statically allocated mutex does not need to be explicitly initialized; by default, a statically allocated mutex is initial- ized with all zeros and its scope is set to be within the calling process. For inter-process synchronization, a mutex needs to be allocated in memory shared between these processes. Since the memory for such a mutex must be allocated dynamically, the mutex needs to be explicitly initialized with the appropriate attribute that indicates inter- process use. Locking and Unlocking A critical section of code is enclosed by a call to lock the mutex and the call to unlock the mutex to protect it from simultaneous access by multiple threads. Only one thread at a time may possess mutually exclusive access to the critical section of code that is enclosed by the mutex-locking call and the mutex-unlocking call, whether the mutex's scope is intra-process or inter-process. A thread calling to lock the mutex either gets exclusive access to the code starting from the successful locking until its call to unlock the mutex, or it waits until the mutex is unlocked by the thread that locked it. Mutexes have ownership, unlike semaphores. Only the thread that locked a mutex, (that is, the owner of the mutex), should unlock it. If a thread waiting for a mutex receives a signal, upon return from the signal handler, the thread resumes waiting for the mutex as if there was no interrupt. Caveats Mutexes are almost like data - they can be embedded in data structures, files, dynamic or static memory, and so forth. Hence, they are easy to introduce into a program. However, too many mutexes can degrade performance and scalability of the application. Because too few mutexes can hinder the concurrency of the application, they should be introduced with care. Also, incorrect usage (such as recursive calls, or violation of locking order, and so forth) can lead to deadlocks, or worse, data inconsistencies. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |MT-Level |MT-Safe | +-----------------------------+-----------------------------+ SEE ALSO
mmap(2), shmop(2), mutex_destroy(3C), mutex_init(3C), mutex_lock(3C), mutex_trylock(3C), mutex_unlock(3C), pthread_create(3C), pthread_mutex_destroy(3C), pthread_mutex_init(3C), pthread_mutex_lock(3C), pthread_mutex_trylock(3C), pthread_mutex_unlock(3C), pthread_mutexattr_init(3C), attributes(5), standards(5) NOTES
In the current implementation of threads, pthread_mutex_lock(), pthread_mutex_unlock(), mutex_lock() mutex_unlock(), pthread_mutex_try- lock(), and mutex_trylock() do not validate the mutex type. Therefore, an uninitialized mutex or a mutex with an invalid type does not return EINVAL. Interfaces for mutexes with an invalid type have unspecified behavior. By default, if multiple threads are waiting for a mutex, the order of acquisition is undefined. USYNC_THREAD does not support multiple mappings to the same logical synch object. If you need to mmap() a synch object to different loca- tions within the same address space, then the synch object should be initialized as a shared object USYNC_PROCESS for Solaris, and PTHREAD_PROCESS_PRIVATE for POSIX. SunOS 5.10 20 Jul 1998 mutex(5)
All times are GMT -4. The time now is 06:51 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy