Sponsored Content
Top Forums UNIX for Advanced & Expert Users How to read a fast written log file at Real time speed? Post 302936629 by achenle on Thursday 26th of February 2015 07:51:49 PM
Old 02-26-2015
That's not really an easy problem to solve. Any solution that meets your requirements has to take into account specifics of the OS, the file system, and even the physical hardware. Obviously such a solution isn't going to be very portable.

The first thing you have to figure out is what's fast enough? Because there's always going to be some delay in reading data from the file after it's been written.

You might be better served interposing something into the logging stream that splits the stream into two - one to the original logging system and one into your real-time parser.
This User Gave Thanks to achenle For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

log users real time

hi.... how i can configurator a log file on real time....on unix solaris.... thanks a lot.... Best Regards... (3 Replies)
Discussion started by: chanfle
3 Replies

2. Shell Programming and Scripting

Real time log file redirect

Hi all, i would like to write the shell script program, it can monitor the access_log "real time" when the access_log writing the line contain "abcdef" the program will be "COPY" this line into a file named "abcdef.txt", do the same thing if the contain "123456" "COPY" it into a file named... (3 Replies)
Discussion started by: eric_wong_ch
3 Replies

3. Shell Programming and Scripting

Perl or Shell script to read a transaction log in real time

Hello, I have a Apache webserver running on RedHat. Its primary function is a proxy server for users accessing the internet. I have a transaction log that logs every transactions of every users. For users trying to access certain sites/content the transactions goes into a 302 redirect loop and... (2 Replies)
Discussion started by: bruno406
2 Replies

4. Programming

Read/Write a fairly large amount of data to a file as fast as possible

Hi, I'm trying to figure out the best solution to the following problem, and I'm not yet that much experienced like you. :-) Basically I have to read a fairly large file, composed of "messages" , in order to display all of them through an user interface (made with QT). The messages that... (3 Replies)
Discussion started by: emitrax
3 Replies

5. Filesystems, Disks and Memory

data from blktrace: read speed V.S. write speed

I analysed disk performance with blktrace and get some data: read: 8,3 4 2141 2.882115217 3342 Q R 195732187 + 32 8,3 4 2142 2.882116411 3342 G R 195732187 + 32 8,3 4 2144 2.882117647 3342 I R 195732187 + 32 8,3 4 2145 ... (1 Reply)
Discussion started by: W.C.C
1 Replies

6. Shell Programming and Scripting

Read/Search file being written to giving error due to timing issues

The following is a piece of code to rename LOG_FILE_NEW to LOG_FILE once you get a result (either RUNNING or SHUTDOWN) RESULT="" sleep 30 while ; do sleep 10 RESULT=`sed -n '/RUNNING/'p ${LOG_FILE_NEW}` if ; then RESULT=`sed -n '/SHUTTING_DOWN/'p ${LOG_FILE_NEW}` fi done mv... (3 Replies)
Discussion started by: sonorous
3 Replies

7. HP-UX

HP-UX real time audit log writing

Hey all, I have a problem I was hoping to get some help on. So I have my two auditfiles, audfile1 and audfile2 that can be written to, I want to have the text version of them write to an NFS mount that I have set up. So i already know that i can do .secure/etc/audsp audfile1 > //nfsmount/folder/... (5 Replies)
Discussion started by: CleverRiver6
5 Replies

8. Shell Programming and Scripting

[solved] How to see log in real time?

Hi people I have a bash script with a line like this: python example.py >> log & But i can't see anything in the log file while python program is running only if the program ends seems to write the log file. "$ cat log" for example don't show anything until the program ends. Is there... (4 Replies)
Discussion started by: Tieso
4 Replies

9. Shell Programming and Scripting

Archiving or removing few data from log file in real time

Hi, I have a log file that gets updated every second. Currently the size has grown to 20+ GB. I need to have a command/script, that will try to get the actual size of the file and will remove 50% of the data that are in the log file. I don't mind removing the data as the size has grown to huge... (8 Replies)
Discussion started by: Souvik Patra
8 Replies

10. Shell Programming and Scripting

Log all the commands input by user at real time in /var/log/messages

Below is my script to log all the command input by any user to /var/log/messages. But I cant achieve the desired output that i want. PLease see below. function log2syslog { declare COMMAND COMMAND=$(fc -ln -0) logger -p local1.notice -t bash -i -- "$USER:$COMMAND" } trap... (12 Replies)
Discussion started by: invinzin21
12 Replies
sgels.f(3)							      LAPACK								sgels.f(3)

NAME
sgels.f - SYNOPSIS
Functions/Subroutines subroutine sgels (TRANS, M, N, NRHS, A, LDA, B, LDB, WORK, LWORK, INFO) SGELS solves overdetermined or underdetermined systems for GE matrices Function/Subroutine Documentation subroutine sgels (characterTRANS, integerM, integerN, integerNRHS, real, dimension( lda, * )A, integerLDA, real, dimension( ldb, * )B, integerLDB, real, dimension( * )WORK, integerLWORK, integerINFO) SGELS solves overdetermined or underdetermined systems for GE matrices Purpose: SGELS solves overdetermined or underdetermined real linear systems involving an M-by-N matrix A, or its transpose, using a QR or LQ factorization of A. It is assumed that A has full rank. The following options are provided: 1. If TRANS = 'N' and m >= n: find the least squares solution of an overdetermined system, i.e., solve the least squares problem minimize || B - A*X ||. 2. If TRANS = 'N' and m < n: find the minimum norm solution of an underdetermined system A * X = B. 3. If TRANS = 'T' and m >= n: find the minimum norm solution of an undetermined system A**T * X = B. 4. If TRANS = 'T' and m < n: find the least squares solution of an overdetermined system, i.e., solve the least squares problem minimize || B - A**T * X ||. Several right hand side vectors b and solution vectors x can be handled in a single call; they are stored as the columns of the M-by-NRHS right hand side matrix B and the N-by-NRHS solution matrix X. Parameters: TRANS TRANS is CHARACTER*1 = 'N': the linear system involves A; = 'T': the linear system involves A**T. M M is INTEGER The number of rows of the matrix A. M >= 0. N N is INTEGER The number of columns of the matrix A. N >= 0. NRHS NRHS is INTEGER The number of right hand sides, i.e., the number of columns of the matrices B and X. NRHS >=0. A A is REAL array, dimension (LDA,N) On entry, the M-by-N matrix A. On exit, if M >= N, A is overwritten by details of its QR factorization as returned by SGEQRF; if M < N, A is overwritten by details of its LQ factorization as returned by SGELQF. LDA LDA is INTEGER The leading dimension of the array A. LDA >= max(1,M). B B is REAL array, dimension (LDB,NRHS) On entry, the matrix B of right hand side vectors, stored columnwise; B is M-by-NRHS if TRANS = 'N', or N-by-NRHS if TRANS = 'T'. On exit, if INFO = 0, B is overwritten by the solution vectors, stored columnwise: if TRANS = 'N' and m >= n, rows 1 to n of B contain the least squares solution vectors; the residual sum of squares for the solution in each column is given by the sum of squares of elements N+1 to M in that column; if TRANS = 'N' and m < n, rows 1 to N of B contain the minimum norm solution vectors; if TRANS = 'T' and m >= n, rows 1 to M of B contain the minimum norm solution vectors; if TRANS = 'T' and m < n, rows 1 to M of B contain the least squares solution vectors; the residual sum of squares for the solution in each column is given by the sum of squares of elements M+1 to N in that column. LDB LDB is INTEGER The leading dimension of the array B. LDB >= MAX(1,M,N). WORK WORK is REAL array, dimension (MAX(1,LWORK)) On exit, if INFO = 0, WORK(1) returns the optimal LWORK. LWORK LWORK is INTEGER The dimension of the array WORK. LWORK >= max( 1, MN + max( MN, NRHS ) ). For optimal performance, LWORK >= max( 1, MN + max( MN, NRHS )*NB ). where MN = min(M,N) and NB is the optimum block size. If LWORK = -1, then a workspace query is assumed; the routine only calculates the optimal size of the WORK array, returns this value as the first entry of the WORK array, and no error message related to LWORK is issued by XERBLA. INFO INFO is INTEGER = 0: successful exit < 0: if INFO = -i, the i-th argument had an illegal value > 0: if INFO = i, the i-th diagonal element of the triangular factor of A is zero, so that A does not have full rank; the least squares solution could not be computed. Author: Univ. of Tennessee Univ. of California Berkeley Univ. of Colorado Denver NAG Ltd. Date: November 2011 Definition at line 183 of file sgels.f. Author Generated automatically by Doxygen for LAPACK from the source code. Version 3.4.1 Sun May 26 2013 sgels.f(3)
All times are GMT -4. The time now is 11:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy