How to copy a binary file while the file is being written to by another process


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users How to copy a binary file while the file is being written to by another process
# 1  
Old 03-01-2012
How to copy a binary file while the file is being written to by another process

Hello,
Can I copy a binary file while the file is being written to by another process?

Another process (program) “P1” creates and opens (for writing) binary file “ABC” on local disk. Process P1 continuously write into ABC file every couple of seconds, adding 512-byte blocks of data. ABC file usually grows to the size of 20-30 MB.
I need to make a copy (replica) of ABC file every one hour, while process P1 is still active. My copy action must not disturb process P1 and no data can be lost (not written) into ABC file because of my copy action. It is ok if my action causes a short delay in P1 writing, but data must not be lost. It is ok if my copy of ABC file misses the latest blocks of data from ABC file. If I can safely make a copy of ABC file which will be a snapshot of ABC file from 5 minutes ago, that is ok for me.

Can you please help me with the description of steps that I need to do in my program or script?
Is there a way to open a file for read-only, but in a way that UNIX would guarantee that my reading program will not disturb the other writing program P1?


I'm working on AIX server.

Thank you,
Milan
# 2  
Old 03-02-2012
Reading from a file doesn't disturb anything, at all. The writer doesn't know or care anything else is reading, and the data being written or position it's being written to won't be upset, reset, or altered.

The real worry would be whether you'd get anything sensible by doing so. You could catch the data file in an in-between or invalid state -- a half-written line or some such -- except you've handily told us your application appends, so that earlier records won't be altered; and does so in individual 512-byte blocks.

If your application writes 512 bytes at a time, i.e. one write of 512 bytes instead of 512 writes of 1 byte, you'll never catch the file in a halfway state -- from a user perspective, the file will grow cleanly in 512-byte jumps.

In short, it ought to be safe to simply copy this file. You don't even need to stop the application.

Last edited by Corona688; 03-02-2012 at 12:25 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to copy a file from one server to anther server and execute the binary

Hi , Is there any script to copy a files (weblogic bianary + silent.xml ) from one server (linux) to another servers and then execute the copy file. We want to copy a file on multiple servers and run the installation. Thanks (1 Reply)
Discussion started by: Nawrajesh
1 Replies

2. Shell Programming and Scripting

Convert binary file to csv and then back to the binary format

Hello *nix specialists, Im working for a non profit organisation in Germany to transport DSL over WLAN to people in areas without no DSL. We are using Linksys WRT 54 router with DD-WRT firmware There are at the moment over 180 router running but we have to change some settings next time. So my... (7 Replies)
Discussion started by: digidax
7 Replies

3. Shell Programming and Scripting

Fiding file except file name written in another file

HI All, Actually i have file xxx.txt where entry is like as follow. prepaid_123.txt prepaid_134.txt prepaid_156.txt So my problem is i want to find all fille except these three file in current directly of same format prepaid_xxx.txt Can you please help Thanks in Adcance (7 Replies)
Discussion started by: sujit_kashyap
7 Replies

4. Shell Programming and Scripting

Notification as popups when a file being written

I would like to know Is it possible to get a notification as pop-up in linux when a folder with extension '.aqs' written a popup should come as " The folder has been written " Thank you in advance (5 Replies)
Discussion started by: bal_nair
5 Replies

5. UNIX for Dummies Questions & Answers

12. If an ‘88’ Record with BAI Code ‘902’ was found on input file and not written to Output file, re

This is my input file like this 03,105581,,015,+00000416418,,,901,+00000000148,,,922,+00000000354,,/ 49,+00000000000416920,00002/ 03,5313236,,015,+00231036992,,,045,+00231036992,,,901,+00000048428,,/ 88,100,+0000000000000,0000000,,400,+0000000000000,0000000,/ 88,902,+0000000079077,,/... (0 Replies)
Discussion started by: sgoud
0 Replies

6. Shell Programming and Scripting

check if some file is in copy process, then transfer it

my user copy large files, and it's take 10min for file to be copied to the server (/tmp/user/ files/), if in the meantime start my scheduled script, then it will copy a part of some file to server1 my idea is to check the file size twice in a short period (1-2 seconds) of time, then compare, if... (5 Replies)
Discussion started by: waso
5 Replies

7. Shell Programming and Scripting

Ensure file copy is complete before starting process

Hi experts, I have a requirement wherein a user is uploading a file to the Landing directory on one of our Linux servers. A cron job is scheduled to run after every 5 minutes which will pick up the files from the source (Landing) dir and copy to the target dir, and once successfully copied to... (4 Replies)
Discussion started by: adi_2_chaos
4 Replies

8. Solaris

compiled binary file gives "cannot execute binary file"

Hi, I have two Solaris machines. 1. SunOS X 5.8 Generic_108528-29 sun4u sparc SUNW,Sun-Blade-1500 2. SunOS Y 5.8 Generic_108528-13 sun4u sparc SUNW,Ultra-60 I am trying to buiild a project on both these machines. The Binary output file compiled on machine 2 runs on both the machines. Where... (0 Replies)
Discussion started by: scgupta
0 Replies

9. Shell Programming and Scripting

Checking a file is not being written to

Hello All I am attempting to write a shell script (bourne shell script) which will copy a tar'd and compressed file from a directory to a staging area but will not know whether the file is still open for write since files are being ftp's to my site at random times during the day. Once I am... (14 Replies)
Discussion started by: kanejm
14 Replies

10. UNIX for Dummies Questions & Answers

File being used/written

Hello, Which command in unix can tell whether a file is being used/written by another process. e.g. If one process is copying a very big file in some directory and there is another cronjob process which checks for a new file and in this directory and process the file. I want to check, if the... (4 Replies)
Discussion started by: sanjay92
4 Replies
Login or Register to Ask a Question