Sponsored Content
Top Forums UNIX for Dummies Questions & Answers How to scan only new lines added in file? Post 70370 by RTM on Thursday 28th of April 2005 08:26:10 AM
Old 04-28-2005
You can find out how many lines are in the log file with wc -l. Use a second wc -l on the file to find out how many more lines were added - then just tail the difference. Save your last wc -l into a separate file for use on the next run

Run this command to start it all off (change the location and name of file to suit your needs)
wc -l logfile > /tmp/logfile.linecount

Get the number of lines from the linecount file (use cut or awk to get only the first field). Use that number and a new wc -l of the log file to find out how many new lines. Do your processing against only the new lines utilizing the tail command (see the man page). Save the new number of lines to your linecount file for the next run.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File Scan

Hi everyone , i m working on Sun solaris and i have a file "smsapp.cur" which has information like this paragraph given below , there are millions of such paragraphs From:923212802736 To:923222326807 logMessage: 07-04-08 17:34:29 Getting message topup from code page default in language English... (2 Replies)
Discussion started by: Dastard
2 Replies

2. Shell Programming and Scripting

File Inventory Scan

Is there such a script out there that will Analyze a folder and subfolders, getting file type utilization and file aging. A CSV file will be created with file information File,Size,Date Created,Last Modified,Last Accessed,Extension,File Type,Owner A Summary report text file will also be created... (4 Replies)
Discussion started by: seacros
4 Replies

3. Shell Programming and Scripting

Perl script to scan back lines

Hi Perl gurus, I have this file to scan through. Sample lines below: 2008031A, USERNAME, 12345, give ABC, take XYZ, transaction submitted 2008031B, USERNAME, 12346, waiting for processing 2008031C, USERNAME, 12347, Retrieving response 2008031D, USERNAME, 12348, This is not a valid dealing... (3 Replies)
Discussion started by: gholdbhurg
3 Replies

4. Shell Programming and Scripting

Get the latest added part of a text file?

Hi, I want to get the latest added part of a text file sent over the network to for analysis. Is there a tool to use to keep track of which part of a text file that has already been analysed so only the "new" lines will be sent and handled? I have checked a few tools but I still donīt know how to... (3 Replies)
Discussion started by: pcrs
3 Replies

5. Shell Programming and Scripting

Scan a log file

hi guys, i am very new to scripting & lookin for a shell script/perl script which would scan another file with the keyword "no change" & take a count of the same. Let me know if any further details are required. (3 Replies)
Discussion started by: nhanda
3 Replies

6. Shell Programming and Scripting

how to check if a new line has been added to a file?

Have come up with the following but it doesn't seem to work.. Is there some other command i could use to get this to work? OUTPATH=/home/out PARMFILE=$OUTPATH/jobcount_test.txt LOG=$OUTPATH/job_count_monthlymail_log.txt HLOG=$OUTPATH/job_count_monthlymail_hlog.txt # echo " started at... (2 Replies)
Discussion started by: Jazmania
2 Replies

7. Shell Programming and Scripting

I want Trailer to be added into the text file.

Hi folks, I want Trailer to be added into the txt file the format is below. flatfile-> abc.txt count of the file is 500 records. I want the trailer in this format: TRAILER|500 (pipe delimeter). Please suggest the comands ASAP. Rgds Ann (5 Replies)
Discussion started by: Haque123
5 Replies

8. Shell Programming and Scripting

Log File Scan

I need to read the last line of a log file and save it, sleep for X minutes and read the last line again. If the line is the same, exit 1, otherwise sleep for X minutes until the last line contains 'Status: Process completed'. Can anyone offer advice here? Thanks. (2 Replies)
Discussion started by: mode09
2 Replies

9. Shell Programming and Scripting

Scan a file in realtime and execute certain commands on encountering 5 consecutive identical lines

Mysql log has something like below: I need a bash shell script that will do the following: 1) The script will scan the mysql.log file constantly in real time (something like tail -F mysql.log) 2) If it encounters 5 consecutive identical lines then it would invoke some commands (say... (4 Replies)
Discussion started by: proactiveaditya
4 Replies

10. Shell Programming and Scripting

Error after I added several lines

I have added several lines to my code let cnt=0 if then if then cnt=`expr ${cnt} + 1` sftp_name=EOMMSRPT1 echo "EOMsftpname = ${sftp_name}" >> ${LOGF} ... (2 Replies)
Discussion started by: digioleg54
2 Replies
CG(1)																	     CG(1)

NAME
cg - Recursively grep for a pattern and store it. SYNOPSIS
cg [ -l ] | [ [ -i ] pattern [ files ] ] DESCRIPTION
cg does a search though text files (usually source code) recursively for a pattern, storing matches and displaying the output in a human- readable fashion. It is intended to give some of the functionaly of AT&T's cscope(1) tool, with the advantages of simplicity and not being language-specific. The script will colorize output if configured as such. It is typically run with a Perl regular expression to search for. The search can be made case insensitive by using the -i option. A list of files may also be specified with an additional argument after the pattern. Put the files pattern in quotes to make it be matched by Perl rather than by the shell. Running the script with no arguments will recall the results of the previous search. After the search, entries found can be edited using the vg(1) script. The -l option shows the last log made. SOME EXAMPLES
cg - alone recalls the previous search results. cg -i pattern - search the default list of files for all files matching the pattern (and case-insensitively). cg pattern '*.c' - search recursively for pattern in all *.c files. This automatically converts '*' to '.*' and '.' to '.' for you and does a Perl pattern match on all files in the tree. cg pattern *.c - search through the shell-expanded list of *.c files, so not done recursively (in other words, only the files your shell pass to the script as arguments). cg -l - show the last log made. COMMAND-LINE OPTIONS -i Do a case-insensitive search. -l Show the last log made. -p Toggle the default pager option. cg has a bulit-in pager function, which can be enabled or disabled by default (in .cgvgrc). If the default is enabled, this option disables the pager; if the default is disabled, this option enables it. -P Force the built-in pager to be disabled. FILES
${HOME}/.cglast Log file of the last search. ${HOME}/.cgvgrc Per-user configuration file (if the defaults are not desireable). ${HOME}/.cgvg/* Log files in $HOSTNAME.shell_pid form with the log of the last search. SEE ALSO
vg(1), perl(1), find(1), grep(1), cscope(1) AUTHOR
cg was written by Joshua Uziel <uzi@uzix.org>. 13 Mar 2002 CG(1)
All times are GMT -4. The time now is 02:16 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy