Knowing when a different program modifies a file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Knowing when a different program modifies a file
# 1  
Old 04-17-2013
Knowing when a different program modifies a file

so i was testing something on a test box running linux. i manually vi'ed the /var/log/messages file. and i noticed, the file immediately stopped being updated.

it wasn't until i restarted the syslog process that events started being recorded in it again.

so that tells me, the syslog process knows when that file is modified by a process other than itself, so that prevents it from working.

can someone please show me how i can apply this same tactic to a file of my own?

for instance, if i have a file called /home/skysmart/boldness.txt. I want to know whenever this file is "vi'ed" by a real life user or if someone other than a specific process adds contents to it, as in ">>" /">" .
# 2  
Old 04-17-2013
I'm guessing that you're really saying that new lines being written to the log file did not show up in your vi editing buffer until you closed vi and reloaded the file. It is extremely unlikely that anything stopped writing to a file because vi had that file open.

When you edit a file using vi (or ex or ed or emacs or any other editor) you load a copy of that file into a buffer. You edit the buffer; not the underlying file. If you want to see recent additions to the file while you are editing it, you need to reload the buffer from the file. In vi, the command to reload the buffer is :e. If you have changed the buffer and have not written the updates back to another file; you'll need to use :e!. If you change the file and write those changes back to the file while some other process is writing to it, whether the changes you made to the file or additions added by that other process or some combination of those changes and additions will appear in the file after you exit vi is unspecified.
# 3  
Old 04-17-2013
Quote:
Originally Posted by Don Cragun
I'm guessing that you're really saying that new lines being written to the log file did not show up in your vi editing buffer until you closed vi and reloaded the file. It is extremely unlikely that anything stopped writing to a file because vi had that file open.

When you edit a file using vi (or ex or ed or emacs or any other editor) you load a copy of that file into a buffer. You edit the buffer; not the underlying file. If you want to see recent additions to the file while you are editing it, you need to reload the buffer from the file. In vi, the command to reload the buffer is :e. If you have changed the buffer and have not written the updates back to another file; you'll need to use :e!. If you change the file and write those changes back to the file while some other process is writing to it, whether the changes you made to the file or additions added by that other process or some combination of those changes and additions will appear in the file after you exit vi is unspecified.
i'm pretty familiar with vi. what i was saying was that, apparently, when you vi a file thats being written to by syslog, syslog stops updating that file. yeah, when i vied that filed and saved it, i saw my additions in there. that wasn't the issue. the problem was, after my additions, i expected to have syslog continue to update the log file as usual. but no. it just stopped updating, until i restarted it.

this happened on linux red hat 6.2. i'm guessing most people aren't aware of this?
# 4  
Old 04-17-2013
Try the following, and you will see that while you are editing the file, syslog continues to update the log file.

- go to the end of the file in vi and see what is there.
- wait a few minutes to let something loggable happen.
- in a separate window, do a tail on the log file.
- the results will be different, because syslog continues to update the file.

Here is another demo:
Code:
$ date > date.txt
$ vi date.txt # will see single line
$ date >> date.txt # in another window, while vi open
$ cat date.txt
Wed Apr 17 13:56:14 PDT 2013
Wed Apr 17 13:56:27 PDT 2013

When you saved the log file from within vi, you wiped out the changes that syslog had made. Normally, not a good idea to make changes to syslog log file, unless some over-riding benefit.
# 5  
Old 04-18-2013
syslogd is not "prevented from working", and /var/log/messages is not "stopped being updated".

When you edit that file, AND save it, a new copy is created, of which syslogd does not know. It happily keeps logging to the old file, accessed via inode number, as you can see using the lsof command.
Send a HUP signal to the process to close and reopen all files.
# 6  
Old 04-18-2013
Quote:
Originally Posted by hanson44
When you saved the log file from within vi, you wiped out the changes that syslog had made.
That is incorrect. Whatever changes syslog had made are still there. When saving, vim unlinks the original file and creates a new one. syslog is still working with the original, as RudiC points out.

While the original file is no longer reachable through the filesystem, any process with an open descriptor to the original's contents can still read/write from/to it. Only when the last of those descriptor's is closed will the kernel remove the unreachable file.

From the POSIX unlink(2) manual:
Quote:
When the file's link count becomes 0 and no process has the file open, the space occupied by the file shall be freed and the file shall no longer be accessible. If one or more processes have the file open when the last link is removed, the link shall be removed before unlink() returns, but the removal of the file contents shall be postponed until all references to the file are closed.
To demonstrate this, let's use sed to delete all empty lines from a file, without using a temp file (-i, even when available, uses a temp file):
Code:
{ rm file; sed '/./!d' > file; } < file

1) { ... } < file opens a descriptor to the original file contents. As long as this descriptor is open, the original file's contents are accessible.
2) rm file unlinks the file. At this point, the file is no longer reachable through the filesystem hierarchy.
3) The redirection in sed ... > file creates a new file and redirects stdout to it. sed inherits its stdin descriptor from the parent sh, through which it has access to the original file's content.

Such "cleverness" is usually a very bad idea. Not creating the temp file means that, should the system fail at just the right time, you could be left without a reachable version of the data. And even though a temp file isn't created, the amount of storage required is the same (the original version of the file and the version without empty lines will coexist for some finite amount of time).

If instead an editor which does not unlink the original file were used, e.g. ed, there would then be the problem of multiple unsynchronized writers. The resulting file's contents will be some indeterminate, interleaved melange of data written by multiple processes.

Regards,
Alister
# 7  
Old 04-18-2013
Addendum to the unlinked "old file, accessed via inode number": You could try and undelete the old file using (on linux systems!) debugfs:
man debugfs:
Quote:
undel <inode num> [pathname]
Undelete the specified inode number (which must be surrounded by angle brackets) so that it and its blocks are marked in use,
and optionally link the recovered inode to the specified pathname. The e2fsck command should always be run after using the
undel command to recover deleted files.
The chance of finding all blocks intact is high as the file as such is still there and held open by the syslogd process. In fact, thinking twice, it should be 100%
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Curl to download file from subdivx.com after following location without knowing the file name/extens

This question could be specific to the site subdivx.com In the past, I've been able to download a file following location using cURL but there is something about subdivx.com that's different and can't figure out how to get it to work. I tried the following directly in the terminal with no... (5 Replies)
Discussion started by: MoonD
5 Replies

2. Solaris

Passwd -l or -u modifies lastchg field in /etc/shadow file

Hi, I have a Solaris 10 box where password aging is not functioning properly. Using the passwd command with the -l or -u options causes the lastchg field in the /etc/shadow file to be modified. Therefore, if a user's password is set to expire in 90 days and they are 1 day away, all they have... (4 Replies)
Discussion started by: cschar
4 Replies

3. UNIX Desktop Questions & Answers

Knowing the size and location of variables in a C program

So I need some help with this. Pardon me if I'm posting in the wrong forum, after some googling for my answer and finding nothing I found this forum. It seemed appropriate for what I was seeking. I just didnt find a forum that concerned the use of GDB. I'm learning to use the C language and GDB.... (2 Replies)
Discussion started by: Cambria
2 Replies

4. Emergency UNIX and Linux Support

Is there any way to set the files modified date and stamp to last modifies time?

Actually i did modification in a file on server by mistake, now its showing current time stamp, is there any way to set the files modified date and stamp to last modifies time. Please advice here.Thanks in advance.:b: (7 Replies)
Discussion started by: saluja.deepak
7 Replies

5. Shell Programming and Scripting

how to use the filehandle stored in a variable without knowing its file association

how to use the filehandle stored in a variable without knowing its file association i.e. the filename code my $logFH = $connObj->get('logFH'); infoPrint("Variable is of type IO \n") if(UNIVERSAL::isa($logFH, 'IO')); infoPrint("$logFH\n"); output == INFO :: Variable is of type... (0 Replies)
Discussion started by: rrd1986
0 Replies

6. Shell Programming and Scripting

How Can I get terminal no when someone modifies CRONTAB file

Hi All I have a script which drops a mail with the changes done on crontab file for every 1 hour Can anyone pls tell me how can I modify the script so that it should display the terminal from which the crontab file hasbeen modified. Quick responce much appriciated !! Many thanks in... (7 Replies)
Discussion started by: jagadish_gaddam
7 Replies

7. Shell Programming and Scripting

Knowing whether the file has completely SFTP ed

Hi.. Can Anyone out there help me? I need to write a script to convert a file in EDCIDC format to CSV The files will be transfered through sftp to the box. Is there a way to check the file has finished being transfered or still transfering. so that my conversion task will be performed after... (3 Replies)
Discussion started by: ramukandada
3 Replies

8. UNIX for Dummies Questions & Answers

Rename file knowing the first 7 carachters

Hi, people. I need some help with this: i have this file " PROVEDP_???_yyyymmdd " , and i want to rename to this " IN_PROV_yyyy???.dat " . The " ??? " is the Month , but the file could be created on April, but the name coulb be March, for example.So i need to grab the 3 caracters ... (2 Replies)
Discussion started by: osramos
2 Replies

9. UNIX for Advanced & Expert Users

knowing progress while reading a file

Hi, I am parsing a very big file say 10 MB. It 'll take more than an hour ..I want to know the progress say in % .Is there any way to do that??? or (Is there any way to know which line of the file I am in ) (2 Replies)
Discussion started by: sakthi.abdullah
2 Replies

10. Shell Programming and Scripting

Script that modifies root level files

I run a decent size Solaris 8 network where we use host files and no DNS servers. I have a master host file to push out to all the machines (also would like to do system and services too) but, the only way I've ever been able to do it is buy telneting into the machine and ftping the file into place... (10 Replies)
Discussion started by: turbo90awd
10 Replies
Login or Register to Ask a Question

Featured Tech Videos