11-22-2005
Fail tail algorithm
I am currently working on code that simulates a file tail algorithm since the only way to retrieve the required information is from within a file, and this information needs to be retrieved in as close to real time as possible when the event enters the file. I cannot use system("tail <options>") directly since the file contents have to be read, parsed, and formatted into a different format on the screen.
Currently, the algorithm I am employing is to open the current file (I determine which file is the most recent by constantly polling the date of the files in this directory), "attach" to that file, then read to an end file delimiter, formatting and spitting the output as I go. Once I reach this end file delimiter, I rewind by the length of the delimiter, sleep about 10 ms using nanosleep(), then go back and read again. Unfortunately, the other end user who writes to file doesn't provide any semaphore blocking between the writes, so sometimes my reads don't always grab all the data the first time, and since this file is maintained by another programmer and are unwilling to add a blocking event for me, I've had trouble on/off trying to find the best time between not spiking the CPU, keeping up with the reads fast enough if a lot of data spits through, and also with the on/off problem of not reading all the data in one pass.
I've seemed to notice UNIX's tail has the same problems with not reading in all the data sometimes and some of the lines are not complete depending on the timing, so there doesn't really seem to be any way around that without some kind of synchronization, which I know won't happen since our third party vendors are always very stubborn to change. What I would like to improve though, is the way the file polling is done. While my task doesn't consume a lot of CPU, it still consumes a larger amount than other tasks, and I've tried to use select() which doesn't appear to work on file descriptors for acutal files. Select() never blocks, but keeps returning right away. Is there a "best time" that anyone knows about from UNIX's tail, how long they wait for each poll?
Thanks for your interest in replying:
Chris
9 More Discussions You Might Find Interesting
1. Programming
Hi
I search an exemple of scheduling Feedback algorithm, or help about how to create one.
Thanks (0 Replies)
Discussion started by: messier79
0 Replies
2. Programming
Looking for an algorithm to compute the number of days between two given dates I came across a professor's C program located here: http://cr.yp.to/2001-275/struct1.c
I was wondering if anyone could tell me where the value 678882 in the line
int d = dateday - 678882;
comes from and also the... (1 Reply)
Discussion started by: williamf
1 Replies
3. Programming
what algorithm a FTP application uses i mean whn implemented in socket programming..if you could give a little decription (1 Reply)
Discussion started by: toughguy2handle
1 Replies
4. Shell Programming and Scripting
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
21444 tomusr 213M 61M sleep 29 10 1:20:46 0.1% java/43
21249 root 93M 44M sleep 29 10 1:07:19 0.2% java/56
is there anyway i can use a command to get the total of the SIZE? 306M (Derive from... (5 Replies)
Discussion started by: filthymonk
5 Replies
5. Programming
Hi guys , in my study book from which I re-learn C is task to generate all possible characters combination from numbers entered by the user. I know this algorithm must use combinatorics to calculate all permutations. Problem is how to implement algortihm.
// This program reads the four numbers... (0 Replies)
Discussion started by: solaris_user
0 Replies
6. Programming
Can you help to adjust the void dijkstra(int s) function to find a path from source to every node so that the minimum cost on that path is maximum.
Ex:
From 1 to 2 we have 1 - 3 - 4 - 2 , costs(2+3+4+5)
From 1 to 2 we have 1 - 5 - 6 - 2 , costs(3+3+4+5)
I need the algorithm to choose path 1... (4 Replies)
Discussion started by: ali2011
4 Replies
7. Shell Programming and Scripting
I have 250 files that have 16 columns each - all numbered as follows stat.1000, stat.1001, stat.1002, stat.1003....stat.1250.
I would like to join all 250 of them together tail by tail as follows. For example
stat.1000
a b c
d e f
stat.1001
g h i
j k l
So that my output... (2 Replies)
Discussion started by: kayak
2 Replies
8. Shell Programming and Scripting
I have a requirement of masking few specific fields in the UNIX file. The details are as following-
File is fixed length file with each record of 250 charater length.
2 fields needs to be masked – the positions are 21:30 and 110:120
The character by character making needs to be done which... (5 Replies)
Discussion started by: n78298
5 Replies
9. Shell Programming and Scripting
Redirecting the tail output fails when log rotation happens even though i have used -F. But when i tail and see the output on terminal this does not happen. Note i have also used & to execute this statement in background.
Suppose if i want to tail a file /opt/SAMPLE.txt and redirect its output... (1 Reply)
Discussion started by: CN1
1 Replies
MTAIL(1) User Commands MTAIL(1)
NAME
mtail - tail variant designed for web developers monitoring logfiles
SYNOPSIS
mtail [options] <file>...
DESCRIPTION
MonkeyTail allows a user to tail multiple files on both local and remote hosts and clearly marks inactivity by putting 5 newlines in the
output whenever a pause in output over 3 seconds is detected.
MonkeyTail is implemented a fairly simple wrapper script around standard tail, ssh, and sudo.
OPTIONS
-q Quiet mode
--quiet
" "
-n Output the last N lines of each file before tailing (defaults to 0)
<file>...
Files to tail.
These can specified in the following ways:
@<groupname>
- expands the group (from .mtailrc) to a list of
files to tail
<filename>
- tails a local file.
+<filename> - attempts to sudo and tail a local file (will
prompt for pwd if required).
<remotehost>:<filename>
- attempts to invoke tail via ssh on a remote
host.
+<remotehost>:<filename> - attempts to invoke sudo tail via ssh on a
remote host (will prompt for pwd if required).
SEE ALSO
mtailrc(5), tail(1)
AUTHOR
Martyn Smith <martyn@dollyfish.net.nz>
mtail May 2008 MTAIL(1)