Sponsored Content
Top Forums UNIX for Dummies Questions & Answers How to check if my log file is growing properly? Post 302913876 by jim mcnamara on Wednesday 20th of August 2014 12:22:58 PM
Old 08-20-2014
Some considerations.

Most modern kernels maintain a memory cache of pages written to files. The dirty (changed or added data) pages get written to disk physically - when the system does a fflush (or equivalent) call. Which is not always RIGHT NOW. Also file mtime and atime updates may be deferred until the file descriptor is actually closed. All of this is kernel-dependent and depends what kind of file system is in use (plus mount options).

What this means:
1. Please post your OS name and version
2. the fstab or vfstab (in /etc ) entry for your filesystem.

Why?

The answers given will very probably work for you, but may not always be both timely and correct, which is what you seem to want. Constant runs of RudiC's nice little script may not show you that data was sent by the logging program to the log - until a cache flush occurs. This may be every 30 seconds or so, whenever your sync daemon runs. If you have one. That is why we ask those two questions.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

How to check a file in UNIX is closed or growing?

We have a third party tool in UNIX to kick off a 'file copy' job based on a file existance. If a specific file exists in an UNIX directory, another process should start copy the file into another system for further processing. The issue is, the copy job is starting as soon as the file exists in... (6 Replies)
Discussion started by: kslakshm
6 Replies

2. UNIX for Dummies Questions & Answers

.osm file growing

my /etc/.osm file is growing rapidly and logging large amounts of activity. Can anyone tell me what this file is for and what types of information is logged in this file. Thanks in advance for your help!! (1 Reply)
Discussion started by: golfs4us
1 Replies

3. UNIX for Dummies Questions & Answers

Find out the maximum growing file in a mount

I need to find the file that is growing in the mount. Say yesterday the utilised space was 95% but today that is 96%. How do i find the file that is growing in size. Have checked the same with du/df options but was not able to find much. Please suggest the best possible option. (3 Replies)
Discussion started by: raman1605
3 Replies

4. HP-UX

how to redirect the growing contents of log file to another file in unix

how to redirect the growing contents of log file to another file in unix (2 Replies)
Discussion started by: megh
2 Replies

5. Shell Programming and Scripting

FTP check if file exists and log it

Hi, I need to create a script that checks if a file exists on an offsite server which only has ftp enabled. I was originally going to use perls Net::FTP class but the client does not have perl installed nor wants it. So, I have to use a shell script which logs into the server, then ... (1 Reply)
Discussion started by: gseyforth
1 Replies

6. Solaris

Growing a file system-SVM

Hi gurus Im a newbie in solaris..I need to extend file system space in solaris 10 which is using SVM..I have a file system /pin02 which is 93% full n needs to be extended..only 3.6 gb avail space left..the file system is not mirrored...normal ufs file system only..can u please tel me t... (6 Replies)
Discussion started by: madanmeer
6 Replies

7. UNIX for Dummies Questions & Answers

Find the file which is growing at high speed

Hi, A log file which is growing at high speed, don't know the name of it. How to find the respective file? Many thanks. (2 Replies)
Discussion started by: venkatesht
2 Replies

8. Shell Programming and Scripting

Check log file size every 10 minute. Alert if log not update

How to check log size every 10min. by script (can use crontab) if log size not change with alert "Log not update" Base run on SunOS 5.8 Generic_Virtual sun4u sparc SUNW,SPARC-Enterprise logFiles="log1.log log2.log" logLocation="/usr/home/test/log/" Out put. Tue Jan 31... (3 Replies)
Discussion started by: ooilinlove
3 Replies

9. Shell Programming and Scripting

Error check for copying growing directories

I have a simple script which copies directory from one place to another and deleting the source . I am facing a situation when new files gets added when the script has started running. Its resulting in data loss Please suggest a way to avoid data loss. I googled a lot but most are perl... (11 Replies)
Discussion started by: ningy
11 Replies

10. UNIX for Advanced & Expert Users

Incremental extract from growing log file.

We have a log file which has 16 million row. We want to read all the lines appended from the last time we read using sed command sed -n '<START_LINE>,<LAST_LINE>p' abc.csv I can store this last line line so I can give replace that with START_LINE in my next read. The problem is wc -l which... (2 Replies)
Discussion started by: one2connect
2 Replies
FSYNC(2)						     Linux Programmer's Manual							  FSYNC(2)

NAME
fsync, fdatasync - synchronize a file's in-core state with storage device SYNOPSIS
#include <unistd.h> int fsync(int fd); int fdatasync(int fd); Feature Test Macro Requirements for glibc (see feature_test_macros(7)): fsync(): _BSD_SOURCE || _XOPEN_SOURCE || /* since glibc 2.8: */ _POSIX_C_SOURCE >= 200112L fdatasync(): _POSIX_C_SOURCE >= 199309L || _XOPEN_SOURCE >= 500 DESCRIPTION
fsync() transfers ("flushes") all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor fd to the disk device (or other permanent storage device) where that file resides. The call blocks until the device reports that the transfer has completed. It also flushes metadata information associated with the file (see stat(2)). Calling fsync() does not necessarily ensure that the entry in the directory containing the file has also reached disk. For that an explicit fsync() on a file descriptor for the directory is also needed. fdatasync() is similar to fsync(), but does not flush modified metadata unless that metadata is needed in order to allow a subsequent data retrieval to be correctly handled. For example, changes to st_atime or st_mtime (respectively, time of last access and time of last modi- fication; see stat(2)) do not require flushing because they are not necessary for a subsequent data read to be handled correctly. On the other hand, a change to the file size (st_size, as made by say ftruncate(2)), would require a metadata flush. The aim of fdatasync() is to reduce disk activity for applications that do not require all metadata to be synchronized with the disk. RETURN VALUE
On success, these system calls return zero. On error, -1 is returned, and errno is set appropriately. ERRORS
EBADF fd is not a valid file descriptor open for writing. EIO An error occurred during synchronization. EROFS, EINVAL fd is bound to a special file which does not support synchronization. CONFORMING TO
4.3BSD, POSIX.1-2001. AVAILABILITY
On POSIX systems on which fdatasync() is available, _POSIX_SYNCHRONIZED_IO is defined in <unistd.h> to a value greater than 0. (See also sysconf(3).) NOTES
Applications that access databases or log files often write a tiny data fragment (e.g., one line in a log file) and then call fsync() imme- diately in order to ensure that the written data is physically stored on the harddisk. Unfortunately, fsync() will always initiate two write operations: one for the newly written data and another one in order to update the modification time stored in the inode. If the mod- ification time is not a part of the transaction concept fdatasync() can be used to avoid unnecessary inode disk write operations. If the underlying hard disk has write caching enabled, then the data may not really be on permanent storage when fsync() / fdatasync() return. When an ext2 file system is mounted with the sync option, directory entries are also implicitly synced by fsync(). On kernels before 2.4, fsync() on big files can be inefficient. An alternative might be to use the O_SYNC flag to open(2). In Linux 2.2 and earlier, fdatasync() is equivalent to fsync(), and so has no performance advantage. SEE ALSO
bdflush(2), open(2), sync(2), sync_file_range(2), hdparm(8), mount(8), sync(8), update(8) COLOPHON
This page is part of release 3.25 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. Linux 2008-11-07 FSYNC(2)
All times are GMT -4. The time now is 10:03 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy