Sponsored Content
Full Discussion: Fail tail algorithm
Top Forums Programming Fail tail algorithm Post 90427 by foureightyeast on Tuesday 22nd of November 2005 02:59:06 PM
Old 11-22-2005
Fail tail algorithm

I am currently working on code that simulates a file tail algorithm since the only way to retrieve the required information is from within a file, and this information needs to be retrieved in as close to real time as possible when the event enters the file. I cannot use system("tail <options>") directly since the file contents have to be read, parsed, and formatted into a different format on the screen.

Currently, the algorithm I am employing is to open the current file (I determine which file is the most recent by constantly polling the date of the files in this directory), "attach" to that file, then read to an end file delimiter, formatting and spitting the output as I go. Once I reach this end file delimiter, I rewind by the length of the delimiter, sleep about 10 ms using nanosleep(), then go back and read again. Unfortunately, the other end user who writes to file doesn't provide any semaphore blocking between the writes, so sometimes my reads don't always grab all the data the first time, and since this file is maintained by another programmer and are unwilling to add a blocking event for me, I've had trouble on/off trying to find the best time between not spiking the CPU, keeping up with the reads fast enough if a lot of data spits through, and also with the on/off problem of not reading all the data in one pass.

I've seemed to notice UNIX's tail has the same problems with not reading in all the data sometimes and some of the lines are not complete depending on the timing, so there doesn't really seem to be any way around that without some kind of synchronization, which I know won't happen since our third party vendors are always very stubborn to change. What I would like to improve though, is the way the file polling is done. While my task doesn't consume a lot of CPU, it still consumes a larger amount than other tasks, and I've tried to use select() which doesn't appear to work on file descriptors for acutal files. Select() never blocks, but keeps returning right away. Is there a "best time" that anyone knows about from UNIX's tail, how long they wait for each poll?

Thanks for your interest in replying:
Chris
 

9 More Discussions You Might Find Interesting

1. Programming

Feedback algorithm

Hi I search an exemple of scheduling Feedback algorithm, or help about how to create one. Thanks (0 Replies)
Discussion started by: messier79
0 Replies

2. Programming

Algorithm problem

Looking for an algorithm to compute the number of days between two given dates I came across a professor's C program located here: http://cr.yp.to/2001-275/struct1.c I was wondering if anyone could tell me where the value 678882 in the line int d = dateday - 678882; comes from and also the... (1 Reply)
Discussion started by: williamf
1 Replies

3. Programming

FTP's algorithm

what algorithm a FTP application uses i mean whn implemented in socket programming..if you could give a little decription (1 Reply)
Discussion started by: toughguy2handle
1 Replies

4. Shell Programming and Scripting

algorithm

PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP 21444 tomusr 213M 61M sleep 29 10 1:20:46 0.1% java/43 21249 root 93M 44M sleep 29 10 1:07:19 0.2% java/56 is there anyway i can use a command to get the total of the SIZE? 306M (Derive from... (5 Replies)
Discussion started by: filthymonk
5 Replies

5. Programming

Please help me to develop algorithm

Hi guys , in my study book from which I re-learn C is task to generate all possible characters combination from numbers entered by the user. I know this algorithm must use combinatorics to calculate all permutations. Problem is how to implement algortihm. // This program reads the four numbers... (0 Replies)
Discussion started by: solaris_user
0 Replies

6. Programming

Looking for Your Help on dijkstra algorithm

Can you help to adjust the void dijkstra(int s) function to find a path from source to every node so that the minimum cost on that path is maximum. Ex: From 1 to 2 we have 1 - 3 - 4 - 2 , costs(2+3+4+5) From 1 to 2 we have 1 - 5 - 6 - 2 , costs(3+3+4+5) I need the algorithm to choose path 1... (4 Replies)
Discussion started by: ali2011
4 Replies

7. Shell Programming and Scripting

Joining multiple files tail on tail

I have 250 files that have 16 columns each - all numbered as follows stat.1000, stat.1001, stat.1002, stat.1003....stat.1250. I would like to join all 250 of them together tail by tail as follows. For example stat.1000 a b c d e f stat.1001 g h i j k l So that my output... (2 Replies)
Discussion started by: kayak
2 Replies

8. Shell Programming and Scripting

Masking algorithm

I have a requirement of masking few specific fields in the UNIX file. The details are as following- File is fixed length file with each record of 250 charater length. 2 fields needs to be masked – the positions are 21:30 and 110:120 The character by character making needs to be done which... (5 Replies)
Discussion started by: n78298
5 Replies

9. Shell Programming and Scripting

When redirecting tail -F output fail after log rotation?

Redirecting the tail output fails when log rotation happens even though i have used -F. But when i tail and see the output on terminal this does not happen. Note i have also used & to execute this statement in background. Suppose if i want to tail a file /opt/SAMPLE.txt and redirect its output... (1 Reply)
Discussion started by: CN1
1 Replies
MTAIL(1)							   User Commands							  MTAIL(1)

NAME
mtail - tail variant designed for web developers monitoring logfiles SYNOPSIS
mtail [options] <file>... DESCRIPTION
MonkeyTail allows a user to tail multiple files on both local and remote hosts and clearly marks inactivity by putting 5 newlines in the output whenever a pause in output over 3 seconds is detected. MonkeyTail is implemented a fairly simple wrapper script around standard tail, ssh, and sudo. OPTIONS
-q Quiet mode --quiet " " -n Output the last N lines of each file before tailing (defaults to 0) <file>... Files to tail. These can specified in the following ways: @<groupname> - expands the group (from .mtailrc) to a list of files to tail <filename> - tails a local file. +<filename> - attempts to sudo and tail a local file (will prompt for pwd if required). <remotehost>:<filename> - attempts to invoke tail via ssh on a remote host. +<remotehost>:<filename> - attempts to invoke sudo tail via ssh on a remote host (will prompt for pwd if required). SEE ALSO
mtailrc(5), tail(1) AUTHOR
Martyn Smith <martyn@dollyfish.net.nz> mtail May 2008 MTAIL(1)
All times are GMT -4. The time now is 08:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy